Collaborative Research: RI: Small: Adapting Foundation Models for Multimodal Sequential Decision Making: New Focuses, Formalisms, and Methods

NSF Award Search · 01002425DB NSF RESEARCH & RELATED ACTIVIT · $279,399 · view on nsf.gov ↗

Abstract

Artificial intelligence has achieved remarkable success in recent years, largely driven by advancements in foundation models, which leverage complex neural networks trained on vast amounts of data in order to perform a variety of tasks, such as question answering, text summarization, and image generation. This project seeks to extend the success of foundation models to sequential decision-making, where an agent--a programmable entity---interacts with an environment, seeking to accomplish a task by taking a series of actions over time, with each action influenced by the outcomes of previous actions. Sequential decision-making commonly arises in situations characterized by uncertainty, limited resources, or dynamic conditions, where each decision can have an impact on future actions. The objective is to select a sequence of actions that maximizes profits, rewards, utilities, or some other well-defined objective. Adapting foundation models for sequential decision-making is challenging, because high-quality data is often lacking and it requires recognizing task-specific structures and optimizing long-term objectives, where minor differences can drastically change optimal solutions. This project will develop novel methods for overcoming these challenges to significantly increase the applicability of foundation models for a wide range of sequential decision-making applications, such as smart manufacturing, multi-agent systems, and human-machine interaction. This project will de

Key facts

NSF award ID
2544949
Awardee
Worcester Polytechnic Institute (MA)
SAM.gov UEI
HJNQME41NBU4
PI
Qi Zhang
Primary program
01002425DB NSF RESEARCH & RELATED ACTIVIT
All programs
ROBUST INTELLIGENCE, SMALL PROJECT, EXP PROG TO STIM COMP RES
Estimated total
$279,399
Funds obligated
$277,789
Transaction type
Standard Grant
Period
07/01/2025 → 12/31/2027