# Wearable silent speech technology to enhance impaired oral communication

> **NIH NIH R01** · UNIVERSITY OF TEXAS AT AUSTIN · 2021 · $292,804

## Abstract

Project Summary/Abstract
The long-term objectives of the parent R01 project (R01DC016621, 08/15/2019 – 06/31/2024) are to obtain
a deeper understanding how articulatory movement patterns are mapped to speech particularly when there is
no vocal fold vibration (silent speech) and then to develop a novel, wearable assistive technology called silent
speech interface (SSI) to assist the impaired oral communication for individuals in need (e.g., individuals after
laryngectomy, surgical removal of larynx to treat advanced laryngeal cancer). The parent R01 project aims to
(1) determine the articulatory patterns of normal (vocalized) and silent speech, produced by both healthy
talkers and people after laryngectomy, (2) develop a wearable device for real-time tongue and lip motion
tracking, and (3) synthesize speech from articulation directly. If successful, the proposed research will enhance
human health by making an impact on individuals after laryngectomy and potentially to a broader range of
other speech and voice disorders as well as visual feedback-based secondary language training and speech
therapy.
Through the parent R01 project, we are collecting a unique multi-modal speech dataset from patients following
laryngectomy and healthy controls. This data set includes speech kinematics from multiple tracers attached on
the tongue and the lips, and speech acoustics. Each tracer comprises multiple sensors that measure inertial
and magnetic information that can provide additional information to assist the speech acoustic and the
articulation-to-speech algorithm. Making such data ML ready for others to consume was out of scope due to
required effort and complexity needed to pre-process and synchronize sampling between kinematic and
acoustic data. The specific goal of this supplemental project is to make data AI/ML ready by developing the
pre-processing algorithms needed to generate a set of features, along with proper formatting and labeling, that
can be more easily shared through repositories and used by others to evaluate different ML algorithms. New
ML models that will be tested on these ML-ready shared datasets will significantly advance our capabilities to
translate articulatory motion into speech sounds, which will not only improve the quality of life for people
affected by laryngectomy but also for the millions of individuals living with speech sound disorders such as
Parkinson’s disease, and amyotrophic lateral sclerosis.

## Key facts

- **NIH application ID:** 10412276
- **Project number:** 3R01DC016621-03S1
- **Recipient organization:** UNIVERSITY OF TEXAS AT AUSTIN
- **Principal Investigator:** Jun Wang
- **Activity code:** R01 (R01, R21, SBIR, etc.)
- **Funding institute:** NIH
- **Fiscal year:** 2021
- **Award amount:** $292,804
- **Award type:** 3
- **Project period:** 2019-08-15 → 2024-07-31

## Primary source

NIH RePORTER: https://reporter.nih.gov/project-details/10412276

## Citation

> US National Institutes of Health, RePORTER application 10412276, Wearable silent speech technology to enhance impaired oral communication (3R01DC016621-03S1). Retrieved via AI Analytics 2026-05-25 from https://api.ai-analytics.org/grant/nih/10412276. Licensed CC0.

---

*[NIH grants dataset](/datasets/nih-grants) · CC0 1.0*