Testing the reward prediction error hypothesis with an axiomatic model

Robb B. Rutledge, Mark Dean, Andrew Caplin, Paul Glimcher

Research output: Contribution to journalArticle

Abstract

Neuroimaging studies typically identify neural activity correlated with the predictions of highly parameterized models, like the many reward prediction error (RPE) models used to study reinforcement learning. Identified brain areas might encode RPEs or, alternatively, only have activity correlated with RPE model predictions. Here, we use an alternate axiomatic approach rooted in economic theory to formally test the entire class of RPE models on neural data. We show that measurements of human neural activity from the striatum, medial prefrontal cortex, amygdala, and posterior cingulate cortex satisfy necessary and sufficient conditions for the entire class of RPE models. However, activity measured from the anterior insula falsifies the axiomatic model, and therefore no RPE model can account for measured activity. Further analysis suggests the anterior insula might instead encode something related to the salience of an outcome. As cognitive neuroscience matures and models proliferate, formal approaches of this kind that assess entire model classes rather than specific model exemplars may take on increased significance.

Original languageEnglish (US)
Pages (from-to)13525-13536
Number of pages12
JournalJournal of Neuroscience
Volume30
Issue number40
DOIs
StatePublished - Oct 6 2010

Fingerprint

Reward
Gyrus Cinguli
Amygdala
Prefrontal Cortex
Human Activities
Neuroimaging
Economics
Learning
Brain

ASJC Scopus subject areas

  • Neuroscience(all)

Cite this

Testing the reward prediction error hypothesis with an axiomatic model. / Rutledge, Robb B.; Dean, Mark; Caplin, Andrew; Glimcher, Paul.

In: Journal of Neuroscience, Vol. 30, No. 40, 06.10.2010, p. 13525-13536.

Research output: Contribution to journalArticle

Rutledge, Robb B. ; Dean, Mark ; Caplin, Andrew ; Glimcher, Paul. / Testing the reward prediction error hypothesis with an axiomatic model. In: Journal of Neuroscience. 2010 ; Vol. 30, No. 40. pp. 13525-13536.
@article{1ca58d8282394c96b52e049d3a825f62,
title = "Testing the reward prediction error hypothesis with an axiomatic model",
abstract = "Neuroimaging studies typically identify neural activity correlated with the predictions of highly parameterized models, like the many reward prediction error (RPE) models used to study reinforcement learning. Identified brain areas might encode RPEs or, alternatively, only have activity correlated with RPE model predictions. Here, we use an alternate axiomatic approach rooted in economic theory to formally test the entire class of RPE models on neural data. We show that measurements of human neural activity from the striatum, medial prefrontal cortex, amygdala, and posterior cingulate cortex satisfy necessary and sufficient conditions for the entire class of RPE models. However, activity measured from the anterior insula falsifies the axiomatic model, and therefore no RPE model can account for measured activity. Further analysis suggests the anterior insula might instead encode something related to the salience of an outcome. As cognitive neuroscience matures and models proliferate, formal approaches of this kind that assess entire model classes rather than specific model exemplars may take on increased significance.",
author = "Rutledge, {Robb B.} and Mark Dean and Andrew Caplin and Paul Glimcher",
year = "2010",
month = "10",
day = "6",
doi = "10.1523/JNEUROSCI.1747-10.2010",
language = "English (US)",
volume = "30",
pages = "13525--13536",
journal = "Journal of Neuroscience",
issn = "0270-6474",
publisher = "Society for Neuroscience",
number = "40",

}

TY - JOUR

T1 - Testing the reward prediction error hypothesis with an axiomatic model

AU - Rutledge, Robb B.

AU - Dean, Mark

AU - Caplin, Andrew

AU - Glimcher, Paul

PY - 2010/10/6

Y1 - 2010/10/6

N2 - Neuroimaging studies typically identify neural activity correlated with the predictions of highly parameterized models, like the many reward prediction error (RPE) models used to study reinforcement learning. Identified brain areas might encode RPEs or, alternatively, only have activity correlated with RPE model predictions. Here, we use an alternate axiomatic approach rooted in economic theory to formally test the entire class of RPE models on neural data. We show that measurements of human neural activity from the striatum, medial prefrontal cortex, amygdala, and posterior cingulate cortex satisfy necessary and sufficient conditions for the entire class of RPE models. However, activity measured from the anterior insula falsifies the axiomatic model, and therefore no RPE model can account for measured activity. Further analysis suggests the anterior insula might instead encode something related to the salience of an outcome. As cognitive neuroscience matures and models proliferate, formal approaches of this kind that assess entire model classes rather than specific model exemplars may take on increased significance.

AB - Neuroimaging studies typically identify neural activity correlated with the predictions of highly parameterized models, like the many reward prediction error (RPE) models used to study reinforcement learning. Identified brain areas might encode RPEs or, alternatively, only have activity correlated with RPE model predictions. Here, we use an alternate axiomatic approach rooted in economic theory to formally test the entire class of RPE models on neural data. We show that measurements of human neural activity from the striatum, medial prefrontal cortex, amygdala, and posterior cingulate cortex satisfy necessary and sufficient conditions for the entire class of RPE models. However, activity measured from the anterior insula falsifies the axiomatic model, and therefore no RPE model can account for measured activity. Further analysis suggests the anterior insula might instead encode something related to the salience of an outcome. As cognitive neuroscience matures and models proliferate, formal approaches of this kind that assess entire model classes rather than specific model exemplars may take on increased significance.

UR - http://www.scopus.com/inward/record.url?scp=77957728784&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77957728784&partnerID=8YFLogxK

U2 - 10.1523/JNEUROSCI.1747-10.2010

DO - 10.1523/JNEUROSCI.1747-10.2010

M3 - Article

C2 - 20926678

AN - SCOPUS:77957728784

VL - 30

SP - 13525

EP - 13536

JO - Journal of Neuroscience

JF - Journal of Neuroscience

SN - 0270-6474

IS - 40

ER -