Electrophysiological Signature of Successor Representation (SR)

NIH RePORTER · NIH · P20 · $407,292 · view on reporter.nih.gov ↗

Abstract

Reinforcement learning (RL) is a computationally-disciplined method to model behavior as agents interact with the environment, change it from one state to another, and collect rewards over time. RL allows us to examine different aspects of learning the environment and decision-making. For example, the RL formulation made it possible for researchers to distinguish between different memory systems as people perform a task. To understand how different elements and representations of RL are implemented in the mammalian brain, it is crucial to have representations supported by neurobiology. Successor representation (SR) is a predictive matrix of the occupancy of future environmental states given a specific state. Interestingly, a few recent studies show that cognitive maps, e.g., in the hippocampus, are implemented in an SR-like fashion. Recent evidence indicates that the prefrontal cortex in humans implements SR-like algorithms to calculate distances. In addition, the pattern of mistakes people make supports the involvement of SR in decision-making. However, minimal work has been done on the electrophysiological correlates of learning SR and updating its values. Through a collaboration with Dr. Wael Asaad's non-human primate lab, we propose analyzing behavioral and neuronal spike data from non-human primates navigating a maze towards a reward. Under the mentorship of Dr. Michael Frank, I will examine how the animals use the SR to learn the maze, the reward location, and how they update the SR when the reward location has changed. Understanding neuronal correlates of SR will allow us to examine how neuronal activity supports decision-making through the RL framework.

Key facts

NIH application ID
10663161
Project number
5P20GM103645-10
Recipient
BROWN UNIVERSITY
Principal Investigator
Mohamed Sherif
Activity code
P20
Funding institute
NIH
Fiscal year
2022
Award amount
$407,292
Award type
5
Project period
2013-08-15 → 2025-07-31