# Determining the explanatory utility of computational reinforcement-learning theories of goal-directed and habitual control at behavioral and neural levels

> **NIH NIH R01** · CALIFORNIA INSTITUTE OF TECHNOLOGY · 2021 · $669,592

## Abstract

Determining the explanatory utility of computational reinforcement-learning
theories of goal-directed and habitual control at behavioral and neural levels
PI: Dr. John P. O’Doherty
Institution: California Institute of Technology
PROJECT SUMMARY
Accumulating evidence supports the existence of two distinct systems for guiding action-selection in the brain:
a goal-directed system in which actions are selected with reference to the current incentive value of the
associated goal or outcome, and a habitual system in which actions are selected reflexively, based solely on
their history of past reinforcement. A computational account for these two systems has been formulated in
terms of two distinct variants of computational reinforcement-learning (RL) theory: model-based (MB) vs
model-free (MF) RL. Yet, empirical evidence in support of the proposed correspondence between the
psychological (RDoC level) and computational RL accounts are sparse. Here we aim to comprehensively
address whether the RDoC level constructs of goal-directed and habitual control can be effectively described
by the computational framework of model-based and model-free RL in humans at both behavioral and neural
levels.
 We plan to administer two distinct behavioral tasks designed to discriminate goal-directed from habitual
control and MB from MF control to a large cohort of healthy participants (n=200) and an undifferentiated cohort
of psychiatric patients (n=100). Our participants will perform these tasks while being scanned with fMRI, in
addition to undergoing resting-state fMRI, and diffusion weighted imaging. We will also measure behavioral
traits and states relevant to psychopathology in the same individuals. We will leverage individual differences
across our behavioral, computational and neural measures in order to determine the extent to which the
psychological constructs and computational accounts are best viewed as being one and the same, or whether
by contrast they diverge in theoretically important ways. Should we detect clear differences between the
psychological (RDoC) constructs and computational descriptions on any of the levels of analysis we utilize, this
will motivate an iterative refinement of the computational framework to better approximate the psychological
(RDoC) level constructs, to be accomplished in parallel to the experimental aims. The distinction between
goals and habits and their proposed computational bases are arguably one of the most influential research
topics in computational psychiatry to date, given the hypothesized relevance of these constructs as a means of
capturing various forms of psychiatric dysfunction. Thus, a better understanding of the nature of the
relationship between these constructs, coupled with a process of active refinement of the computational theory
to achieve a much closer correspondence to the psychological constructs, is going to be critical for progress in
this domain.

## Key facts

- **NIH application ID:** 10205983
- **Project number:** 5R01MH121089-03
- **Recipient organization:** CALIFORNIA INSTITUTE OF TECHNOLOGY
- **Principal Investigator:** JOHN P O'DOHERTY
- **Activity code:** R01 (R01, R21, SBIR, etc.)
- **Funding institute:** NIH
- **Fiscal year:** 2021
- **Award amount:** $669,592
- **Award type:** 5
- **Project period:** 2019-08-02 → 2024-05-31

## Primary source

NIH RePORTER: https://reporter.nih.gov/project-details/10205983

## Citation

> US National Institutes of Health, RePORTER application 10205983, Determining the explanatory utility of computational reinforcement-learning theories of goal-directed and habitual control at behavioral and neural levels (5R01MH121089-03). Retrieved via AI Analytics 2026-05-27 from https://api.ai-analytics.org/grant/nih/10205983. Licensed CC0.

---

*[NIH grants dataset](/datasets/nih-grants) · CC0 1.0*