# Decoding inner speech: An AI approach to transcribing thoughts via EEG & EMG

> **NIH NIH R21** · YALE UNIVERSITY · 2020 · $523,600

## Abstract

ABSTRACT
 Losing the capacity to communicate through language has a significant negative impact on a person’s
autonomy, social interactions, occupation, mental health, and overall quality of life. Many people lose the
capacity to speak and write but keep their thinking intact.
 Inner speech is internally and willfully generated, non-articulated verbal thoughts (e.g., reading in
silence). Changes in the activation patterns of the brain’s language-related areas co-occur with inner speech
and can be detected with electroencephalography (EEG). Furthermore, while inner speech doesn’t lead to any
discernible voice sound or articulation, co-occurring low amplitude electrical discharges in the articulatory
muscles can be detected with electromyography (EMG). The information about ongoing inner speech reflected
in electrophysiological signals (EEG and EMG) can be used to transcribe inner speech into text or voice.
 Machine learning algorithms have been used for this purpose, however, the resulting systems have low
accuracy and/or are constrained by very small vocabularies (~10 words). Furthermore, these systems need to
be trained anew for each user, which significantly increases individual data-collection time. The development of
ready-to-use/minimal-training (fine tuning) systems requires large training datasets that algorithms can use to
learn high-level features capable of being transferred between individuals. Unfortunately, to date there are no
available datasets that are large enough to train these systems.
 To tackle these issues, I have assembled a multidisciplinary team of collaborators from Google AI, Yale
linguistics, and Yale Psychiatry to develop a state-of-the-art deep neural network to transcribe inner speech to
text using EEG and EMG signals. This system will incorporate some of the latest advances in artificial
intelligence and data processing developed by Google AI. It will be designed to transcribe phonemes, thus, in
principle, will be able to transcribe any word. Furthermore, we will collect the largest (x120 times) multi-subject
(n=150) electrophysiological (EEG+EMG) inner speech dataset to date (300 hrs. in total) to train the first ready-
to-use/minimal-training inner speech transcriber system.
 The technology resulting from this study has the potential to radically improve the quality of life of
thousands of patients by providing them with a fast method of communicating their verbal thoughts.
Furthermore, by combining this system with one of the many text-to-speech AIs that are currently available, our
system could potentially restore the patients’ capacity to produce conversational speech.

## Key facts

- **NIH application ID:** 10058047
- **Project number:** 1R21EB029607-01A1
- **Recipient organization:** YALE UNIVERSITY
- **Principal Investigator:** Jose A CORTES-BRIONES
- **Activity code:** R21 (R01, R21, SBIR, etc.)
- **Funding institute:** NIH
- **Fiscal year:** 2020
- **Award amount:** $523,600
- **Award type:** 1
- **Project period:** 2020-09-15 → 2024-09-14

## Primary source

NIH RePORTER: https://reporter.nih.gov/project-details/10058047

## Citation

> US National Institutes of Health, RePORTER application 10058047, Decoding inner speech: An AI approach to transcribing thoughts via EEG & EMG (1R21EB029607-01A1). Retrieved via AI Analytics 2026-05-23 from https://api.ai-analytics.org/grant/nih/10058047. Licensed CC0.

---

*[NIH grants dataset](/datasets/nih-grants) · CC0 1.0*