# CAREER: Perceiving an Actionable 3D World

> **NSF 01002829DB NSF RESEARCH & RELATED ACTIVIT** · Carnegie Mellon University (PA) · $599,585

## Abstract

We live in a 3D world, and our perception systems, despite only receiving 2D retinal percepts, can effortlessly understand the underlying 3D structure, e.g., observing a set of stacked dishes, we understand how they support one another. Even more remarkably, we understand not just what it is, but also what can be, e.g., removing a dish in the middle can cause the tower to collapse. While the computer vision community has made impressive advances in developing computational systems that can reconstruct the underlying 3D from visual input, these systems do not understand about actions and their effects in this context. This project will bridge this gap and build perception systems which have an actionable understanding of the 3D world they observe. Such systems can be broadly useful across applications in computer vision, robotics, and mixed reality, e.g., allowing robots to intelligently act and efficiently learn in generic scenarios and helping virtual assistants better understand and guide their user's actions. This project will also contribute to the development of undergraduate and graduate students via research engagements and development of a specialized course on embodied agents, as well as benefit the community at large through dissemination of research and organization of tutorials.

To achieve its goal, this project will make research contributions along three thrusts: a) developing approaches to learn about 2D affordances (what actions can be performed) and world

## Key facts

- **NSF award ID:** 2442282
- **Awardee organization:** Carnegie Mellon University (PA)
- **SAM.gov UEI:** U3NKNFLNQ613
- **PI:** Shubham Tulsiani
- **Primary program:** 01002829DB NSF RESEARCH & RELATED ACTIVIT
- **All programs:** CAREER-Faculty Erly Career Dev, ROBUST INTELLIGENCE
- **Estimated total:** $599,585
- **Funds obligated:** $348,956
- **Transaction type:** Continuing Grant
- **Period:** 06/15/2025 → 05/31/2030

## Primary source

NSF Award Search: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2442282

## Citation

> US National Science Foundation, Award 2442282, CAREER: Perceiving an Actionable 3D World. Retrieved via AI Analytics 2026-06-07 from https://api.ai-analytics.org/grant/nsf/2442282. Licensed CC0.

---

*[NSF Awards dataset](/datasets/nsf-awards) · CC0 1.0*