Collaborative Research: FRR: Enabling Robots with Adaptive Projector-Based AR in Dynamic Environments

NSF Award Search · 01002526DB NSF RESEARCH & RELATED ACTIVIT · $325,437 · view on nsf.gov ↗

Abstract

This research explores projector-based augmented reality (AR) for communication from a robot. Because projected AR has traditionally required relatively flat surfaces to work, this novel approach allows people to view projected images and messages from robots in unstructured environments. In situations like search and rescue, concert halls, and cluttered homes, it can be hard to know what a robot's next actions might be. This approach supports users need to understand the robot's motions and intent when they are moving near people. This research enables robots to communicate by projecting images or messages onto non-flat surfaces. The project uses computer vision approaches to sense the environment and capture the geometry, photometry, and semantics at the same time. It is expected that this will improve communications when the robot is operating near people. This will also allow the robot to provide directions or point to objects without the need for special equipment. The project will do human studies to determine the effectiveness of the communications. This could make robots easier to communicate with in different situations in daily life. The project will directly advance the fields of Human-Robot Interaction, projected-AR, computer vision, and artificial intelligence (AI) by enabling robots to adaptively-project situated visualizations to multiple people at the same time. The proposed solution enables robots to project onto non-flat surfaces with flexible textures

Key facts

NSF award ID
2529207
Awardee
SUNY at Stony Brook (NY)
SAM.gov UEI
M746VC6XMNH9
PI
Arie E Kaufman
Primary program
01002526DB NSF RESEARCH & RELATED ACTIVIT
All programs
Artificial Intelligence (AI), ROBOTICS
Estimated total
$325,437
Funds obligated
$325,437
Transaction type
Standard Grant
Period
09/01/2025 → 08/31/2028