CAREER: Foundations of Semantic Code Understanding by Large Language Models for Software Maintenance

NSF Award Search · 01003031DB NSF RESEARCH & RELATED ACTIVIT · $677,469 · view on nsf.gov ↗

Abstract

Generative Artificial Intelligence (AI) is rapidly transforming software engineering, with a substantial portion of new code being generated by AI. As generating code becomes easier, the bottleneck shifts to maintaining it through testing, debugging, and repairs. Effective software maintenance fundamentally depends on a deep, accurate understanding of software behavior, yet it remains unclear how well current AI models truly understand software. This project systematically studies and improves AI's understanding of code in realistic maintenance settings. The project's novelties are a principled, evidence-driven methodology for characterizing and strengthening AI model's code comprehension beyond ad hoc benchmarks. The project's broader significance and importance are improved reliability and trustworthiness of automated coding assistants, which benefits software developers as well as scientists and engineers who increasingly rely on AI-generated software. The project establishes a foundation for studying, assisting, and advancing Large Language Models' ability to understand code for effective software maintenance. The research first assesses how well contemporary models understand software and develops a framework that automatically generates controlled, unseen, and dynamic proxy tasks as targeted assessments of code understanding. Based on the cataloged weaknesses, the research investigates software maintenance task redesign strategies that accentuate cognitively demanding code and task patterns to enhance model comprehension. To address the challenge of increasing reliance on self-generated training data, which can cause performance plateaus, this work introduces a new usage paradigm that leverages human expertise and traditional program analysis signals to create high-quality learning examples. This allows models to improve code understanding while reducing cognitive effort by reusing prior reasoning. The expected outcomes include robust evaluation methodologi

Key facts

NSF award ID
2541721
Awardee
Virginia Polytechnic Institute and State University (VA)
SAM.gov UEI
QDE5UHE5XD16
PI
Muhammad Ali Gulzar
Primary program
01003031DB NSF RESEARCH & RELATED ACTIVIT
All programs
Artificial Intelligence (AI), CAREER-Faculty Erly Career Dev, SOFTWARE ENG & FORMAL METHODS
Estimated total
$677,469
Funds obligated
$383,998
Transaction type
Continuing Grant
Period
07/01/2026 → 06/30/2031