CAREER: A Full-Stack Science of AI Reliability: Robust Models and Tractable Collaboration

NSF Award Search · 01002627DB NSF RESEARCH & RELATED ACTIVIT · $600,000 · view on nsf.gov ↗

Abstract

Artificial intelligence (AI) systems increasingly assist with decisions in areas such as, medicine, science, and education, but they still make simple reasoning mistakes. Thus, they can be unreliable in high-stakes settings. This project develops a science of AI reliability by identifying why these systems fail and by designing principled methods to make them more dependable, both on their own and when working with humans. The project's novelties are a unified framework that connects model-level robustness to human-AI collaboration, grounded in controlled mathematical environments that isolate real-world failure modes while remaining tractable for rigorous analysis. The approach bridges theory with targeted experimentation to produce insights that transfer to full-scale systems. The project's broader significance and importance are providing scientific foundations for safer deployment of AI in critical applications such as medical diagnosis and producing open-source evaluation tools for the research community. In addition to its technical objectives, this project extends its impact through expanding the Learning Theory Alliance mentorship program, promoting research with minimal computational resources, and developing new graduate and undergraduate courses. At the model level, the research characterizes why transformers, the backbone of modern AI systems, learn brittle shortcuts rather than robust algorithms and develops principled interventions in training data, training algorithms, and inference to make reasoning reliable by design. These include representative training sets that teach robust algorithmic behavior, sample-efficient methods for multi-step reasoning, and formal safeguards against adversarial manipulation. At the interaction level, the research develops a theory of human-AI collaboration under complementary information and imperfect alignment. The resulting protocols are auditable, grounded in verifiable conditions such as calibration, and enable h

Key facts

NSF award ID
2543725
Awardee
University of Pennsylvania (PA)
SAM.gov UEI
GM1XX56LEP58
PI
Surbhi Goel
Primary program
01002627DB NSF RESEARCH & RELATED ACTIVIT
All programs
Artificial Intelligence (AI), CAREER-Faculty Erly Career Dev, ROBUST INTELLIGENCE
Estimated total
$600,000
Funds obligated
$354,579
Transaction type
Continuing Grant
Period
06/01/2026 → 05/31/2031