Model-based influences on humans' choices and striatal prediction errors

Nathaniel D. Daw, Samuel J. Gershman, Ben Seymour, Peter Dayan, Raymond J. Dolan

Research output: Contribution to journalArticle

Abstract

The mesostriatal dopamine system is prominently implicated in model-free reinforcement learning, with fMRI BOLD signals in ventral striatum notably covarying with model-free prediction errors. However, latent learning and devaluation studies show that behavior also shows hallmarks of model-based planning, and the interaction between model-based and model-free values, prediction errors, and preferences is underexplored. We designed a multistep decision task in which model-based and model-free influences on human choice behavior could be distinguished. By showing that choices reflected both influences we could then test the purity of the ventral striatal BOLD signal as a model-free report. Contrary to expectations, the signal reflected both model-free and model-based predictions in proportions matching those that best explained choice behavior. These results challenge the notion of a separate model-free learner and suggest a more integrated computational architecture for high-level human decision-making.

Original languageEnglish (US)
Pages (from-to)1204-1215
Number of pages12
JournalNeuron
Volume69
Issue number6
DOIs
StatePublished - Mar 24 2011

Fingerprint

Choice Behavior
Corpus Striatum
Learning
Dopamine
Decision Making
Magnetic Resonance Imaging
Reinforcement (Psychology)
Ventral Striatum

ASJC Scopus subject areas

  • Neuroscience(all)

Cite this

Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011). Model-based influences on humans' choices and striatal prediction errors. Neuron, 69(6), 1204-1215. https://doi.org/10.1016/j.neuron.2011.02.027

Model-based influences on humans' choices and striatal prediction errors. / Daw, Nathaniel D.; Gershman, Samuel J.; Seymour, Ben; Dayan, Peter; Dolan, Raymond J.

In: Neuron, Vol. 69, No. 6, 24.03.2011, p. 1204-1215.

Research output: Contribution to journalArticle

Daw, ND, Gershman, SJ, Seymour, B, Dayan, P & Dolan, RJ 2011, 'Model-based influences on humans' choices and striatal prediction errors', Neuron, vol. 69, no. 6, pp. 1204-1215. https://doi.org/10.1016/j.neuron.2011.02.027
Daw ND, Gershman SJ, Seymour B, Dayan P, Dolan RJ. Model-based influences on humans' choices and striatal prediction errors. Neuron. 2011 Mar 24;69(6):1204-1215. https://doi.org/10.1016/j.neuron.2011.02.027
Daw, Nathaniel D. ; Gershman, Samuel J. ; Seymour, Ben ; Dayan, Peter ; Dolan, Raymond J. / Model-based influences on humans' choices and striatal prediction errors. In: Neuron. 2011 ; Vol. 69, No. 6. pp. 1204-1215.
@article{21a30cf9ad8047d28138b950e2b9e482,
title = "Model-based influences on humans' choices and striatal prediction errors",
abstract = "The mesostriatal dopamine system is prominently implicated in model-free reinforcement learning, with fMRI BOLD signals in ventral striatum notably covarying with model-free prediction errors. However, latent learning and devaluation studies show that behavior also shows hallmarks of model-based planning, and the interaction between model-based and model-free values, prediction errors, and preferences is underexplored. We designed a multistep decision task in which model-based and model-free influences on human choice behavior could be distinguished. By showing that choices reflected both influences we could then test the purity of the ventral striatal BOLD signal as a model-free report. Contrary to expectations, the signal reflected both model-free and model-based predictions in proportions matching those that best explained choice behavior. These results challenge the notion of a separate model-free learner and suggest a more integrated computational architecture for high-level human decision-making.",
author = "Daw, {Nathaniel D.} and Gershman, {Samuel J.} and Ben Seymour and Peter Dayan and Dolan, {Raymond J.}",
year = "2011",
month = "3",
day = "24",
doi = "10.1016/j.neuron.2011.02.027",
language = "English (US)",
volume = "69",
pages = "1204--1215",
journal = "Neuron",
issn = "0896-6273",
publisher = "Cell Press",
number = "6",

}

TY - JOUR

T1 - Model-based influences on humans' choices and striatal prediction errors

AU - Daw, Nathaniel D.

AU - Gershman, Samuel J.

AU - Seymour, Ben

AU - Dayan, Peter

AU - Dolan, Raymond J.

PY - 2011/3/24

Y1 - 2011/3/24

N2 - The mesostriatal dopamine system is prominently implicated in model-free reinforcement learning, with fMRI BOLD signals in ventral striatum notably covarying with model-free prediction errors. However, latent learning and devaluation studies show that behavior also shows hallmarks of model-based planning, and the interaction between model-based and model-free values, prediction errors, and preferences is underexplored. We designed a multistep decision task in which model-based and model-free influences on human choice behavior could be distinguished. By showing that choices reflected both influences we could then test the purity of the ventral striatal BOLD signal as a model-free report. Contrary to expectations, the signal reflected both model-free and model-based predictions in proportions matching those that best explained choice behavior. These results challenge the notion of a separate model-free learner and suggest a more integrated computational architecture for high-level human decision-making.

AB - The mesostriatal dopamine system is prominently implicated in model-free reinforcement learning, with fMRI BOLD signals in ventral striatum notably covarying with model-free prediction errors. However, latent learning and devaluation studies show that behavior also shows hallmarks of model-based planning, and the interaction between model-based and model-free values, prediction errors, and preferences is underexplored. We designed a multistep decision task in which model-based and model-free influences on human choice behavior could be distinguished. By showing that choices reflected both influences we could then test the purity of the ventral striatal BOLD signal as a model-free report. Contrary to expectations, the signal reflected both model-free and model-based predictions in proportions matching those that best explained choice behavior. These results challenge the notion of a separate model-free learner and suggest a more integrated computational architecture for high-level human decision-making.

UR - http://www.scopus.com/inward/record.url?scp=79952746011&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=79952746011&partnerID=8YFLogxK

U2 - 10.1016/j.neuron.2011.02.027

DO - 10.1016/j.neuron.2011.02.027

M3 - Article

VL - 69

SP - 1204

EP - 1215

JO - Neuron

JF - Neuron

SN - 0896-6273

IS - 6

ER -