# CAREER: Safety, Human-(mis)alignment, and Interaction-awareness in Control

> **NSF 01002526DB NSF RESEARCH & RELATED ACTIVIT** · Carnegie Mellon University (PA) · $500,000

## Abstract

Autonomous control systems in many domains, such as transportation, healthcare, assistance, and manufacturing, operate with humans in increasingly uncertain and interactive environments. Their design introduces unprecedented challenges in ensuring performance, safety, and human alignment due to a broad range of risks. First, existing techniques in risk quantification and control often assume observable state, system dynamics, and in-distribution data with sufficient risk events. However, these assumptions frequently break down in real-world scenarios. Second, inferring and reducing human-perceived risk requires characterizing safe actions from sparse feedback, but existing techniques often cannot handle this complexity. Finally, in nonstationary interactions, control methods that ignore the opponents’ adaptation can unintentionally encourage aggressive or exploitative behaviors by humans. In this proposal, we study how to provide long-term, lifelong assurances against various risks in the control of autonomous systems operating in uncertain and interactive environments. The outcome of this research will be broadly disseminated through seminars and tutorials and integrated into the PI’s classes and K-12 outreach programs. To enrich the learning experience, we will create a virtual game that allows students to understand key concepts through interacting with others. These efforts will train future researchers and engineers, and facilitate the transfer of insights across domains

## Key facts

- **NSF award ID:** 2442948
- **Awardee organization:** Carnegie Mellon University (PA)
- **SAM.gov UEI:** U3NKNFLNQ613
- **PI:** Yorie Nakahira
- **Primary program:** 01002526DB NSF RESEARCH & RELATED ACTIVIT
- **All programs:** Control systems & applications, CAREER-Faculty Erly Career Dev, PROBABILITY, CONTROL SYSTEMS, ROBOTICS, LEARNING & INTELLIGENT SYSTEMS
- **Estimated total:** $500,000
- **Funds obligated:** $500,000
- **Transaction type:** Standard Grant
- **Period:** 07/01/2025 → 06/30/2030

## Primary source

NSF Award Search: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2442948

## Citation

> US National Science Foundation, Award 2442948, CAREER: Safety, Human-(mis)alignment, and Interaction-awareness in Control. Retrieved via AI Analytics 2026-06-07 from https://api.ai-analytics.org/grant/nsf/2442948. Licensed CC0.

---

*[NSF Awards dataset](/datasets/nsf-awards) · CC0 1.0*