# Language, Modality and the Brain

> **NIH NIH R01** · SAN DIEGO STATE UNIVERSITY · 2024 · $565,705

## Abstract

Project Summary
Sign language is vital to the lives and well-being of many deaf people in the United States who rely on this
mode of communication. However, current theories in linguistics, psychology, and cognitive neuroscience have
all been developed primarily from investigations of spoken language. The focus on speech leaves open many
critical questions about linguistic phenomena that are fundamentally shaped by the visual-manual modality.
NIDCD recognizes these gaps in our understanding – the Strategic Plan states, ”Enhancing our understanding
of visual-manual language systems opens a window into general human cognition (pg. 32).” This project aims
to develop a neurobiological model of sign language perception and comprehension that takes into account
essential modality-specific phenomena: phonology without sound, lexical iconicity (non-arbitrary mappings
between the form and meaning of signs), and `spatial syntax' (the use of locations in signing space to express
grammatical roles and co-reference). Aim 1 of the project is to map the cortical representation of phonological
structure in American Sign Language (ASL) using functional Magnetic Resonance Imaging (fMRI). fMRI
adaptation methods will be used to investigate whether body-selective neural regions become tuned to
linguistic handshapes and body locations in signers. In a second fMRI study, Phonological Neighborhood
Density in ASL will be manipulated to identify neural areas that support lexical-level phonological
representations. Aim 2 of the project is to identify the impact of iconicity on lexical representations and
learning. Event-Related Potentials (ERPs) will be used to assess whether effects of iconicity are task
dependent (e.g., related to the use of picture stimuli) or reflect distinct neural representations for iconic signs
(e.g., more robust encoding of sensory-motor semantic features). ERPs will also be used to investigate
whether brain responses to iconic signs change with learning and whether learners represent signs as
wholistic gestures or are sensitive to internal phonological structure. Aim 3 of the project is to identify the
neural network involved in comprehending `spatial syntax'. fMRI will be used to determine whether right
parietal cortex is engaged when understanding ASL verbs that are directed toward locations in signing space
(R-loci) to indicate grammatical roles. A second fMRI experiment is designed to identify the neural correlates of
spatial indexing (the establishment and maintenance of R-loci). Overall, the project aims to enhance our
understanding of the neurobiology of visual-manual language, which will provide a translational foundation for
treating injury to the language system and for diagnosing language impairments in deaf individuals.

## Key facts

- **NIH application ID:** 10746785
- **Project number:** 5R01DC010997-44
- **Recipient organization:** SAN DIEGO STATE UNIVERSITY
- **Principal Investigator:** KAREN EMMOREY
- **Activity code:** R01 (R01, R21, SBIR, etc.)
- **Funding institute:** NIH
- **Fiscal year:** 2024
- **Award amount:** $565,705
- **Award type:** 5
- **Project period:** 1979-07-01 → 2025-11-30

## Primary source

NIH RePORTER: https://reporter.nih.gov/project-details/10746785

## Citation

> US National Institutes of Health, RePORTER application 10746785, Language, Modality and the Brain (5R01DC010997-44). Retrieved via AI Analytics 2026-05-23 from https://api.ai-analytics.org/grant/nih/10746785. Licensed CC0.

---

*[NIH grants dataset](/datasets/nih-grants) · CC0 1.0*