# CAREER: Responsiveness to Heterogeneous Preferences in AI Alignment and Deployment

> **NSF 01003031DB NSF RESEARCH & RELATED ACTIVIT** · Cornell University (NY) · $598,252

## Abstract

The range of users of artificial intelligence (AI) systems has vastly broadened in recent years, and decisions made with the help of AI are now affecting almost everyone. Naturally, the individuals have different preferences over how the AI should behave, which poses new challenges because traditional approaches for developing and deploying AI assume that users have the same preferences. This project builds a theoretical understanding of the blind spots of current AI development and deployment pipelines by studying, for example, when disagreements can make the AI system adopt an unbalanced position or behave erratically. In their place, the project designs new algorithms for the alignment, fine-tuning, and deployment of AI that are provably robust to disagreeing user preferences and strike a sensible compromise between them. To bring the benefits of this research to the public, the project will also produce two software tools: a website comparing AI models to help users choose the right AI tool for their needs and an AI-enhanced tool providing live information on group opinions in participatory processes. The project also develops new courses and educational materials training students in the mathematical foundations of preference-aware AI.

The project will achieve these goals by integrating key concepts from computational social choice theory (which provides methods for aggregating the individual group members into a collective decision) and generative AI. First, the project analyzes alignment methods through the lens of distortion, which measures the fraction of choice lost when an algorithm observes only pairwise, stochastic comparisons rather than users' true preferences. This analysis quantifies the shortcomings of the dominant alignment method, reinforcement learning from human feedback, and identifies alternatives with better guarantees. Second, the project designs alignment methods satisfying proportionality guarantees adapted from social choice theory su

## Key facts

- **NSF award ID:** 2543553
- **Awardee organization:** Cornell University (NY)
- **SAM.gov UEI:** G56PUALJ3KT5
- **PI:** Paul Goelz
- **Primary program:** 01003031DB NSF RESEARCH & RELATED ACTIVIT
- **All programs:** Artificial Intelligence (AI), CAREER-Faculty Erly Career Dev
- **Estimated total:** $598,252
- **Funds obligated:** $367,129
- **Transaction type:** Continuing Grant
- **Period:** 07/01/2026 → 06/30/2031

## Primary source

NSF Award Search: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2543553

## Citation

> US National Science Foundation, Award 2543553, CAREER: Responsiveness to Heterogeneous Preferences in AI Alignment and Deployment. Retrieved via AI Analytics 2026-05-20 from https://api.ai-analytics.org/grant/nsf/2543553. Licensed CC0.

---

*[NSF Awards dataset](/datasets/nsf-awards) · CC0 1.0*