# Vision in Natural Tasks > **NIH NIH R01** · UNIVERSITY OF TEXAS AT AUSTIN · 2022 · $370,291 ## Abstract Summary/Abstract In the context of natural behavior, humans make continuous sequences of sensory-motor decisions to satisfy current behavioral goals, and vision must provide the information needed to achieve those goals. The proposed work examines gaze and walking decisions in locomotion in outdoor environments, taking advantage of our novel system for measuring combined eye and body movements in these contexts. Currently we have only limited understanding of the constituent tasks in natural locomotion, or the requisite information, and the proposal attempts to specify these. in the context of natural gait, the patterns of optic flow are unexpectedly complex, raising questions about its role. The patterns of motion on the retina during locomotion depend critically on both eye and body motion, and these in turn depend on behavioral goals. Our first Aim is therefore to comprehensively describe the statistics of retinal motion patterns in a variety of terrains and task contexts. We will measure binocular eye and body movements while walking in outdoor terrains of varying roughness, crossing a busy intersection, and making coffee. These contexts will induce different gaze patterns. We will provide a comprehensive description of the motion stimulus in natural locomotion and help separate out self-motion signals from externally generated motion. These data will allow a more precise specification of the response patterns in cortical motion sensitive areas. Because of the complexity of natural motion patterns, we will re-examine the influence of optic flow on walking direction in a virtual reality environment and test alternative explanations for the role of flow. A central task in walking is foot placement, and we will focus on identifying the image properties that make a good foothold. Stereo, structure from motion, and spatial image structure are all likely contenders. We directly investigate the role of stereo in foothold selection by examining gait patterns in stereo-deficient subjects in terrains with varying degrees of roughness. Using a different strategy, we will attempt to predict gaze locations and footholds in rough terrain using convolution neural nets (CNN’s) to identify potential search templates for footholds in rough terrain. We will describe fixation patterns from crosswalk and sidewalk navigation and attempt to make inferences about their purpose, and use Modular Inverse Reinforcement Learning (MIRL) to predict direction decisions and decompose the behavior into sub-tasks. The collection of integrated gaze, body kinematics, and scene images in a range of natural environments is innovative, as little comparable data exists The work will be strengthened by the investigation of stereo- deficient subjects for whom there is almost no integrated eye and body data. Since much of the work in robotics has no visual input at all this should help in development of visual guidance for robots and also help better define the necessary informatio... ## Key facts - **NIH application ID:** 10471328 - **Project number:** 5R01EY005729-35 - **Recipient organization:** UNIVERSITY OF TEXAS AT AUSTIN - **Principal Investigator:** Mary M Hayhoe - **Activity code:** R01 (R01, R21, SBIR, etc.) - **Funding institute:** NIH - **Fiscal year:** 2022 - **Award amount:** $370,291 - **Award type:** 5 - **Project period:** 1984-07-01 → 2024-08-31 ## Primary source NIH RePORTER: https://reporter.nih.gov/project-details/10471328 ## Citation > US National Institutes of Health, RePORTER application 10471328, Vision in Natural Tasks (5R01EY005729-35). Retrieved via AI Analytics 2026-05-24 from https://api.ai-analytics.org/grant/nih/10471328. Licensed CC0. --- *[NIH grants dataset](/datasets/nih-grants) · CC0 1.0*