CAREER: Safety, Human-(mis)alignment, and Interaction-awareness in Control

NSF Award Search · 01002526DB NSF RESEARCH & RELATED ACTIVIT · $500,000 · view on nsf.gov ↗

Abstract

Autonomous control systems in many domains, such as transportation, healthcare, assistance, and manufacturing, operate with humans in increasingly uncertain and interactive environments. Their design introduces unprecedented challenges in ensuring performance, safety, and human alignment due to a broad range of risks. First, existing techniques in risk quantification and control often assume observable state, system dynamics, and in-distribution data with sufficient risk events. However, these assumptions frequently break down in real-world scenarios. Second, inferring and reducing human-perceived risk requires characterizing safe actions from sparse feedback, but existing techniques often cannot handle this complexity. Finally, in nonstationary interactions, control methods that ignore the opponents’ adaptation can unintentionally encourage aggressive or exploitative behaviors by humans. In this proposal, we study how to provide long-term, lifelong assurances against various risks in the control of autonomous systems operating in uncertain and interactive environments. The outcome of this research will be broadly disseminated through seminars and tutorials and integrated into the PI’s classes and K-12 outreach programs. To enrich the learning experience, we will create a virtual game that allows students to understand key concepts through interacting with others. These efforts will train future researchers and engineers, and facilitate the transfer of insights across domains

Key facts

NSF award ID
2442948
Awardee
Carnegie Mellon University (PA)
SAM.gov UEI
U3NKNFLNQ613
PI
Yorie Nakahira
Primary program
01002526DB NSF RESEARCH & RELATED ACTIVIT
All programs
Control systems & applications, CAREER-Faculty Erly Career Dev, PROBABILITY, CONTROL SYSTEMS, ROBOTICS, LEARNING & INTELLIGENT SYSTEMS
Estimated total
$500,000
Funds obligated
$500,000
Transaction type
Standard Grant
Period
07/01/2025 → 06/30/2030