# Closing the loop on markerless object tracking

> **NIH NIH R03** · BROWN UNIVERSITY · 2020 · $162,500

## Abstract

Abstract/Summary
Tracking the movements of objects and parts of objects - referred to as pose estimation - is critical for
understanding the mechanisms underlying complex behavior. Characterizing dynamic behaviors of animals
(and other systems) is central to many disciplines, including computer science, physics, ethology, kinesiology,
and sports medicine. Here we focus on neuroscience, where linking brain activity to associated dynamic
behaviors is critical for both understanding normal function as well as effects of injury, disease, or degeneration.
Invasive methods for measuring behavior are highly accurate, but require placement of sensors that may
themselves interact with behavior and which may be susceptible to deterioration or infection. Video provides a
non-invasive approach to characterizing behavior over time. Extracting behavior from video streams has,
historically, been a slow and laborious process. Recent work in machine learning and artificial neural networks
(ANNs), though, has revolutionized this process, making the analysis of complex video far easier and more
accurate. While these systems are highly flexible, they were not designed for real time use, meaning that large
video files must first be stored to disk for subsequent analysis. This poses two problems that this proposal will
attempt to address. First, there is significant cost and management challenges associated with storage of large
video stores, forming a practical barrier for adoption of this important technology for characterizing behavior.
Second, estimates related to behavioral state are not available in real time so they cannot be used to control the
experiment. We will develop a research methodology for “closing the loop”, by taking the networks trained by
an existing and highly successful markerless object tracking system (DeepLabCut) and optimizing them for real
time inference. After the system is functional, verified, and benchmarked, it will be shared with the community
through open source repositories.

## Key facts

- **NIH application ID:** 10047656
- **Project number:** 1R03MH123990-01
- **Recipient organization:** BROWN UNIVERSITY
- **Principal Investigator:** DAVID L SHEINBERG
- **Activity code:** R03 (R01, R21, SBIR, etc.)
- **Funding institute:** NIH
- **Fiscal year:** 2020
- **Award amount:** $162,500
- **Award type:** 1
- **Project period:** 2020-06-01 → 2023-05-31

## Primary source

NIH RePORTER: https://reporter.nih.gov/project-details/10047656

## Citation

> US National Institutes of Health, RePORTER application 10047656, Closing the loop on markerless object tracking (1R03MH123990-01). Retrieved via AI Analytics 2026-05-22 from https://api.ai-analytics.org/grant/nih/10047656. Licensed CC0.

---

*[NIH grants dataset](/datasets/nih-grants) · CC0 1.0*