Collaborative Research: RI: Medium: Using Systematic Relationships and Phonetic Speech Foundation Models for Universal Speech-to-Text across Varieties of Languages

NSF Award Search · 01002526DB NSF RESEARCH & RELATED ACTIVIT · $821,902 · view on nsf.gov ↗

Abstract

This project introduces a new way to develop speech-to-text systems for language varieties in which little digital data is available, especially when a related variety already has plenty of digital data. Languages differ from one another, but they also show a great deal of variation within themselves. People in different regions or countries often pronounce words in the same language differently. For example, many people in the southern US pronounce the “i” in words like “ride” as an “a” sound (similar to the one found in “rad”), while most other Americans do not. Most speech-to-text systems—which turn spoken words into written text—focus on just one variety of each language. However, in everyday life, many people speak other varieties. Existing speech-to-text systems often do not work well for these varieties. Improving speech recognition for low-resource varieties represents a business opportunity, one that can help more Americans access voice-powered tools and services. This project takes one step towards this goal. It innovates by leveraging the fact that the differences between various varieties of the same language often follow predictable patterns. For example, since the pronunciations of words in different regions change following rules that apply to the whole vocabulary, one can often predict how a word will be pronounced in one variety if one knows how it is pronounced in another. The project will develop a powerful AI model (POWSM) that can transcribe pronunciation

Key facts

NSF award ID
2504019
Awardee
Carnegie Mellon University (PA)
SAM.gov UEI
U3NKNFLNQ613
PI
David R Mortensen
Primary program
01002526DB NSF RESEARCH & RELATED ACTIVIT
All programs
ROBUST INTELLIGENCE, MEDIUM PROJECT
Estimated total
$821,902
Funds obligated
$821,902
Transaction type
Standard Grant
Period
09/01/2025 → 08/31/2029