CAREER: Towards Proprioceptive Vision Through the Lens of Wearable Cameras

NSF Award Search · 01002829DB NSF RESEARCH & RELATED ACTIVIT · $597,457 · view on nsf.gov ↗

Abstract

Understanding how humans interact with the physical world is essential for teaching intelligent systems to perform complex tasks effectively. Everyday activities, such as grasping a coffee mug and taking a sip, often involve the seamless integration of visual perception and motor control, a process that current intelligent systems struggle to replicate. This project aims to bridge the gap by developing a new family of visual sensing and learning approaches to track hand motion and interpret daily interactions, all using wearable cameras. Further, this project will demonstrate a key application in occupational safety and health by adapting the technology to assess injury risks in the workplace. By advancing the sensing and analysis of human interactions, this research will deepen the understanding of human intelligent behavior, drive the development of more capable intelligent systems, and expand practical applications of such systems. The project’s outputs, including open-source algorithms and hardware platforms, will be disseminated through public competitions, online courses, and diverse outreach activities. This project will advocate a new paradigm of proprioceptive vision, where an intelligent system integrates physical awareness with visual perception to understand how its actions relate to its observations. The project’s goal will be achieved through two interconnected research thrusts. Thrust 1 will develop computer vision approaches to harness wrist-mounted, spatia

Key facts

NSF award ID
2442739
Awardee
University of Wisconsin-Madison (WI)
SAM.gov UEI
LCLSJAGTNZQ7
PI
Yin Li
Primary program
01002829DB NSF RESEARCH & RELATED ACTIVIT
All programs
CAREER-Faculty Erly Career Dev, ROBUST INTELLIGENCE
Estimated total
$597,457
Funds obligated
$350,714
Transaction type
Continuing Grant
Period
07/01/2025 → 06/30/2030