# AUDIOVISUAL INTEGRATION, AGE, AND HEARING LOSS

> **NIH NIH R01** · WASHINGTON UNIVERSITY · 2021 · $609,206

## Abstract

ABSTRACT
People with hearing loss might benefit greatly if they could use the visual signal to recognize
speech. Sadly, as people age, they often experience a decline in both their hearing ability and
their lipreading ability, so that at the very time their audiovisual speech perception could
maximally benefit from visual speech information, they are least able to utilize it. Understanding
the linkage between these phenomena has been a longstanding challenge for both cognitive
science and aural rehabilitation, yet we believe this linkage provides a major clue to
understanding both individual and age-related differences in everyday speech perception. Most
researchers assume that there is a distinct integration stage and that integrative ability
diminishes with age. Our most recent findings, however, suggest that differences in unimodal
performance, and not differences in integrative abilities, underlie both individual and age-related
differences in audiovisual speech perception. Accordingly, the proposed research takes the
unique approach of focusing on both the causes of individual differences in lipreading skill and
on lipreading as a cause of differences in audiovisual speech perception. For Specific Aim 1, we
will use our new feature analysis methods to predict audiovisual speech perception based on
participants' unimodal (i.e., auditory-only and vision-only) performance. We will evaluate the
relative contributions of cognitive, perceptual, and speech production abilities as well as gaze
behavior to both individual and age-related differences in vision-only and audiovisual speech
perception. For Specific Aim 2, we will use a modification of our speech detection and lipread-
yourself tasks, along with fMRI, to assess differences in the correspondence between the
phonetic representations that support speech production and perception. Finally, for Aim 3, we
will identify neural correlates of vision-only and audiovisual speech perception using fMRI. The
results from Aims 1 and 2 will help us identify the regions of interest. We will focus on premotor
areas because we hypothesize that the linkage between speech production and visual speech
perception is critical for understanding how people recognize visual speech signals, and we will
directly compare the activation in premotor cortex during both speech production and perception
using representational similarity analysis, an elegant multi-variate analytic technique. By the end
of the grant cycle, our goal is to have developed a unified neurocognitive framework for
understanding both individual and age-related differences in lipreading and audiovisual speech
perception. This framework and our experimental results will also provide evidence-based
guidance for both speech perception training and aural rehabilitation counselling.

## Key facts

- **NIH application ID:** 10197095
- **Project number:** 5R01DC016594-04
- **Recipient organization:** WASHINGTON UNIVERSITY
- **Principal Investigator:** Nancy Ann Tye-Murray
- **Activity code:** R01 (R01, R21, SBIR, etc.)
- **Funding institute:** NIH
- **Fiscal year:** 2021
- **Award amount:** $609,206
- **Award type:** 5
- **Project period:** 2018-07-01 → 2023-06-30

## Primary source

NIH RePORTER: https://reporter.nih.gov/project-details/10197095

## Citation

> US National Institutes of Health, RePORTER application 10197095, AUDIOVISUAL INTEGRATION, AGE, AND HEARING LOSS (5R01DC016594-04). Retrieved via AI Analytics 2026-05-21 from https://api.ai-analytics.org/grant/nih/10197095. Licensed CC0.

---

*[NIH grants dataset](/datasets/nih-grants) · CC0 1.0*