# Wearable silent speech technology to enhance impaired oral communication

> **NIH NIH R01** · UNIVERSITY OF TEXAS AT AUSTIN · 2020 · $580,870

## Abstract

Project Summary/Abstract
The long-term objectives of this project are to obtain a deeper understanding how articulatory movement
patterns are mapped to speech particularly when there is no vocal fold vibration (silent speech) and then to
develop a novel, wearable assistive technology called silent speech interface (SSI) to assist the impaired oral
communication for individuals in need (e.g., individuals after laryngectomy, surgical removal of larynx to treat
advanced laryngeal cancer). Designed for daily use, the SSI contains a wearable magnetic device and a small
camera for tongue and lip motion tracking, respectively, and an articulation-to-speech synthesizer to output
natural sounding speech that preserves the speaker’s voice characteristics. Specific Aims of the proposal
include to (1) determine the articulatory patterns of normal (vocalized) and silent speech, produced by both
healthy talkers and people after laryngectomy, (2) develop a wearable, wireless magnetic device for real-time
tongue and lip motion tracking, and (3) synthesize speech from articulation directly.
There are currently limited alternative communication options for people who have undergone laryngectomy.
These options include esophageal speech, tracheo-esophageal speech, and use of an artificial larynx (or
electrolarynx). These solutions are either invasive or difficult to use, and all of them result in a hoarse or
mechanical/robotic sounding voice, which can be difficult to understand. In contrast, the SSI in this application
is non-invasive, easy-to-use, and produces natural sounding speech and may even preserve the patient’s
voice identity. We have exciting preliminary results that support the feasibility of the project including that (1)
we have recently developed a wireless magnetic device for tongue motion, and (2) we have demonstrated real-
time articulation-to-speech synthesis with a 90% word accuracy (judged by a human listener). In this project,
we will further reduce the size of the wireless device and make it wearable and conduct articulation-to-speech
algorithms by studying 30 participants after laryngectomy and 30 age- and gender-matched healthy controls. If
successful, the proposed research will enhance human health by making an impact on individuals after
laryngectomy and potentially to a broader range of other speech and voice disorders. In addition, the
technology will have an impact to the speech science field by providing a fist-time-ever tool for potential large-
scale tongue motion data collection and have a variety of broader implications including visual feedback-based
secondary language training and speech therapy, which may benefit millions of people with motor speech
deficits in the United States.

## Key facts

- **NIH application ID:** 9994877
- **Project number:** 5R01DC016621-02
- **Recipient organization:** UNIVERSITY OF TEXAS AT AUSTIN
- **Principal Investigator:** Jun Wang
- **Activity code:** R01 (R01, R21, SBIR, etc.)
- **Funding institute:** NIH
- **Fiscal year:** 2020
- **Award amount:** $580,870
- **Award type:** 5
- **Project period:** 2019-08-15 → 2024-07-31

## Primary source

NIH RePORTER: https://reporter.nih.gov/project-details/9994877

## Citation

> US National Institutes of Health, RePORTER application 9994877, Wearable silent speech technology to enhance impaired oral communication (5R01DC016621-02). Retrieved via AI Analytics 2026-05-23 from https://api.ai-analytics.org/grant/nih/9994877. Licensed CC0.

---

*[NIH grants dataset](/datasets/nih-grants) · CC0 1.0*