# Prolonging Functional Speech in Persons with Amyotrophic Lateral Sclerosis: A Real-Time Virtual Vocal Tract

> **NIH NIH K24** · MGH INSTITUTE OF HEALTH PROFESSIONS · 2021 · $189,937

## Abstract

People with ALS eventually and inevitably experience serious speech impairment due to progressive
deterioration of brain cells that control movements of the tongue, lips and jaw. Despite the
devastating consequences of this speech impairment on quality of life and survival, few options are
available to assist impaired oral communication, and many existing speech-generating technologies
are slow to operate and cost prohibitive. This project seeks to improve quality of life for persons with
impaired speech due to ALS by testing the effectiveness of a low-cost, speech-generating device (a
virtual vocal tract) that could significantly prolong the ability of these patients to communicate orally. If
successful, these techniques could be extended for use by patients' with a broad range of speech
motor impairments.
The virtual vocal track uses machine learning algorithms to predict what a person is attempting to
say, in real-time, based solely on lip movements. Users of the device are able to trigger the playback
of a number of predetermined phrases by simply attempting to articulate what they want to say. Our
previous work has shown the feasibility of this approach using cost-prohibitive laboratory systems
such as electromagnetic articulography. Recent advances in 3D depth mapping camera technology
allow these techniques to be tested for the first time using technologies, which are low-cost, portable
and already being integrated into consumer devices such as laptops and cellphones.
To this end, the system under development will be tested in 60 patients with ALS, representing a
range of speech impairment from normal to severe speech intelligibility (15 normal, 15 mild, 15
moderate, 15 severe). During testing, participants will be cued to articulate the phrases in a random
order as fast as is comfortable for them. The entire session will be recorded and the following
variables will be measured offline: recognition accuracy, recognition latency, task time, % completion,
and communication rate (words per minute). Users will rate the usability and acceptability of the
virtual vocal tract immediately following device testing, using the System Usability Scale. Results of
this testing will be used to address the following specific aims: (1) Determine the accuracy and
latency of real-time phrase synthesis based on dysarthric speech using the virtual vocal tract, (2)
Determine the usability and acceptability of real-time phrases produced using the virtual vocal tract,
and (3) Identify the articulatory and speech factors that degrade recognition accuracy.

## Key facts

- **NIH application ID:** 10201558
- **Project number:** 5K24DC016312-05
- **Recipient organization:** MGH INSTITUTE OF HEALTH PROFESSIONS
- **Principal Investigator:** JORDAN R GREEN
- **Activity code:** K24 (R01, R21, SBIR, etc.)
- **Funding institute:** NIH
- **Fiscal year:** 2021
- **Award amount:** $189,937
- **Award type:** 5
- **Project period:** 2017-07-10 → 2024-06-30

## Primary source

NIH RePORTER: https://reporter.nih.gov/project-details/10201558

## Citation

> US National Institutes of Health, RePORTER application 10201558, Prolonging Functional Speech in Persons with Amyotrophic Lateral Sclerosis: A Real-Time Virtual Vocal Tract (5K24DC016312-05). Retrieved via AI Analytics 2026-05-22 from https://api.ai-analytics.org/grant/nih/10201558. Licensed CC0.

---

*[NIH grants dataset](/datasets/nih-grants) · CC0 1.0*