# Neural Codes Underlying Visual Segmentation

> **NIH NIH R01** · UNIVERSITY OF WISCONSIN-MADISON · 2020 · $449,999

## Abstract

Project Summary/Abstract
 In natural vision, it is rare to encounter an isolated object presented on a blank background. Instead, natural
scenes are often complex and contain multiple entities. Image segmentation refers to the process of partitioning
visual scenes into distinct objects and surfaces, which includes segmenting a figure from the background (figure-
ground segregation) and segmenting multiple objects/surfaces from each other. Segmentation is a fundamental
function of vision and is a gateway to perception, recognition and visually guided action. However, the neural
underpinning of segmentation remains to be understood. A key question is to understand how the brain
represents multiple visual stimuli such that information regarding individual stimuli can be extracted from the
activity of populations of neurons. We address this question in the proposed project to elucidate the neural
mechanisms underlying segmentation and the principles of coding sensory information in neuronal populations.
Visual motion and depth provide potent cues for segmentation. Therefore we focus on understanding how the
brain uses motion and depth cues to achieve segmentation. We have made substantial progress in defining how
middle-temporal (MT) cortex, an area important for motion and depth processing, represents multiple overlapping
visual stimuli. We found that MT neurons show various types of response biases toward one component of
multiple stimuli, revealing a set of novel rules by which multiple stimuli interact within neurons’ receptive fields.
These physiological findings together with our preliminary data on natural scene statistics led us to hypothesize
that the visual system exploits the statistical regularities in natural scenes that differentiate figure from the
background and represents multiple visual stimuli efficiently to achieve segmentation. To test this overarching
hypothesis, we will integrate the approaches of natural scene statistics, neurophysiology, and theoretical
consideration of optimal coding. Specifically, we will characterize natural scene statistics of depth and motion
pertinent to image segmentation, elucidate the functional roles of stereoscopic depth in figure-ground
segregation, define the rules by which neurons in area MT represent multiple spatially-separated stimuli, which
are commonly encountered in natural vision, and determine the signal transformation across multiple brain areas
in the dorsal visual pathway to achieve segmentation. Finally, we will use an Information-Maximization approach
to determine whether the neural representation of multiple visual stimuli is optimal for segmentation. The
proposed study rigorously explores the interaction of multiple stimuli and is expected to provide important insight
into how the visual system solves the challenging problem of segmentation in natural vision.

## Key facts

- **NIH application ID:** 9974166
- **Project number:** 2R01EY022443-06A1
- **Recipient organization:** UNIVERSITY OF WISCONSIN-MADISON
- **Principal Investigator:** Xin Huang
- **Activity code:** R01 (R01, R21, SBIR, etc.)
- **Funding institute:** NIH
- **Fiscal year:** 2020
- **Award amount:** $449,999
- **Award type:** 2
- **Project period:** 2014-01-01 → 2025-06-30

## Primary source

NIH RePORTER: https://reporter.nih.gov/project-details/9974166

## Citation

> US National Institutes of Health, RePORTER application 9974166, Neural Codes Underlying Visual Segmentation (2R01EY022443-06A1). Retrieved via AI Analytics 2026-05-22 from https://api.ai-analytics.org/grant/nih/9974166. Licensed CC0.

---

*[NIH grants dataset](/datasets/nih-grants) · CC0 1.0*