# Attention-guided speech enhancement for hearing impaired listeners

> **NIH NIH SC1** · NEW MEXICO STATE UNIVERSITY LAS CRUCES · 2020 · $222,000

## Abstract

Project Summary/Abstract
The goal of this project is to develop, implement, and evaluate an assistive hearing system that isolates sound
sources that are located in the direction of the user’s eye gaze. The system is meant to be used in conjunction
with a binaural hearing aid to improve speech perception for the hearing impaired population. Poor speech
perception is a frequent complaint of those suffering from hearing loss and is associated with an increased risk
of dementia, depression, and feelings of isolation in older adults. Perceiving a speech stream in the presence
of competing speech is often especially difficult for hearing impaired listeners, and traditional hearing aids do
not do much to solve the problem. Previous attempts to address this issue have often relied upon directional
microphones that are oriented to pick up sounds coming from directly in front of the user. However, the focus
of attention often deviates away from directly in front of the listener, especially in auditory environments with
many sound sources. Instead of a system focused on a fixed location, we propose a highly innovative
directional listening system that uses a microphone array to isolate sounds coming from the focus of the user’s
attention. The major hypotheses of this work are that eye gaze direction is a strong indicator of the focus of
auditory attention, and that a gaze-controlled directional listening system represents an effortless and effective
way to isolate the speech signals that the listener cares about. The project’s first specific aim is to refine an
existing and highly promising directional listening algorithm to maximize its ability to isolate a target speech
signal. Human subjects testing will be used to evaluate many different revisions of the algorithm according to
the speech fidelity provided and listening effort required. The highest-performing version of the algorithm will
be implemented as part of the gaze-controlled system. Concurrent with this effort, the project’s second specific
aim is to determine how gaze information can be best used to control the directional listening system. More
generally, the goal is to determine how the focus of visual attention corresponds to the focus of auditory
attention in normal and hearing impaired subjects. This aim includes a series of basic research studies in
which eye movements are monitored while subjects view complex audiovisual speech stimuli. These data will
suggest the best way to translate eye tracking data into an estimate of the focus of auditory attention, while
also providing results relevant to the multisensory attention literature. Finally, the major goal of third specific
aim is to conduct a comprehensive test of the gaze-controlled directional listening system with hearing
impaired subjects. Subjects will complete listening tasks in auditory and visual environments of varying
complexity. Longer-term testing will also be completed to evaluate system performance with experienced
users...

## Key facts

- **NIH application ID:** 9838739
- **Project number:** 5SC1DC016452-04
- **Recipient organization:** NEW MEXICO STATE UNIVERSITY LAS CRUCES
- **Principal Investigator:** Justin MacDonald
- **Activity code:** SC1 (R01, R21, SBIR, etc.)
- **Funding institute:** NIH
- **Fiscal year:** 2020
- **Award amount:** $222,000
- **Award type:** 5
- **Project period:** 2017-01-06 → 2022-12-31

## Primary source

NIH RePORTER: https://reporter.nih.gov/project-details/9838739

## Citation

> US National Institutes of Health, RePORTER application 9838739, Attention-guided speech enhancement for hearing impaired listeners (5SC1DC016452-04). Retrieved via AI Analytics 2026-06-24 from https://api.ai-analytics.org/grant/nih/9838739. Licensed CC0.

---

*[NIH grants dataset](/datasets/nih-grants) · CC0 1.0*