Surprise! Neural correlates of Pearce-Hall and Rescorla-Wagner coexist within the brain

Matthew R. Roesch, Guillem R. Esber, Jian Li, Nathaniel D. Daw, Geoffrey Schoenbaum

Research output: Contribution to journalArticle

Abstract

Learning theory and computational accounts suggest that learning depends on errors in outcome prediction as well as changes in processing of or attention to events. These divergent ideas are captured by models, such as Rescorla-Wagner (RW) and temporal difference (TD) learning on the one hand, which emphasize errors as directly driving changes in associative strength, vs. models such as Pearce-Hall (PH) and more recent variants on the other hand, which propose that errors promote changes in associative strength by modulating attention and processing of events. Numerous studies have shown that phasic firing of midbrain dopamine (DA) neurons carries a signed error signal consistent with RW or TD learning theories, and recently we have shown that this signal can be dissociated from attentional correlates in the basolateral amygdala and anterior cingulate. Here we will review these data along with new evidence: (i) implicating habenula and striatal regions in supporting error signaling in midbrain DA neurons; and (ii) suggesting that the central nucleus of the amygdala and prefrontal regions process the amygdalar attentional signal. However, while the neural instantiations of the RW and PH signals are dissociable and complementary, they may be linked. Any linkage would have implications for understanding why one signal dominates learning in some situations and not others, and also for appreciating the potential impact on learning of neuropathological conditions involving altered DA or amygdalar function, such as schizophrenia, addiction or anxiety disorders. Published 2012. This article is a U.S. Government work and is in the public domain in the USA.

Original languageEnglish (US)
Pages (from-to)1190-1200
Number of pages11
JournalEuropean Journal of Neuroscience
Volume35
Issue number7
DOIs
StatePublished - Apr 2012

Fingerprint

Learning
Brain
Dopaminergic Neurons
Mesencephalon
Habenula
Corpus Striatum
Public Sector
Gyrus Cinguli
Anxiety Disorders
Dopamine
Schizophrenia

Keywords

  • Amygdala
  • Attention
  • Dopamine
  • Learning
  • Prediction error

ASJC Scopus subject areas

  • Neuroscience(all)

Cite this

Surprise! Neural correlates of Pearce-Hall and Rescorla-Wagner coexist within the brain. / Roesch, Matthew R.; Esber, Guillem R.; Li, Jian; Daw, Nathaniel D.; Schoenbaum, Geoffrey.

In: European Journal of Neuroscience, Vol. 35, No. 7, 04.2012, p. 1190-1200.

Research output: Contribution to journalArticle

Roesch, Matthew R. ; Esber, Guillem R. ; Li, Jian ; Daw, Nathaniel D. ; Schoenbaum, Geoffrey. / Surprise! Neural correlates of Pearce-Hall and Rescorla-Wagner coexist within the brain. In: European Journal of Neuroscience. 2012 ; Vol. 35, No. 7. pp. 1190-1200.
@article{330c43d30e8f41c5953bdde31ab8341b,
title = "Surprise! Neural correlates of Pearce-Hall and Rescorla-Wagner coexist within the brain",
abstract = "Learning theory and computational accounts suggest that learning depends on errors in outcome prediction as well as changes in processing of or attention to events. These divergent ideas are captured by models, such as Rescorla-Wagner (RW) and temporal difference (TD) learning on the one hand, which emphasize errors as directly driving changes in associative strength, vs. models such as Pearce-Hall (PH) and more recent variants on the other hand, which propose that errors promote changes in associative strength by modulating attention and processing of events. Numerous studies have shown that phasic firing of midbrain dopamine (DA) neurons carries a signed error signal consistent with RW or TD learning theories, and recently we have shown that this signal can be dissociated from attentional correlates in the basolateral amygdala and anterior cingulate. Here we will review these data along with new evidence: (i) implicating habenula and striatal regions in supporting error signaling in midbrain DA neurons; and (ii) suggesting that the central nucleus of the amygdala and prefrontal regions process the amygdalar attentional signal. However, while the neural instantiations of the RW and PH signals are dissociable and complementary, they may be linked. Any linkage would have implications for understanding why one signal dominates learning in some situations and not others, and also for appreciating the potential impact on learning of neuropathological conditions involving altered DA or amygdalar function, such as schizophrenia, addiction or anxiety disorders. Published 2012. This article is a U.S. Government work and is in the public domain in the USA.",
keywords = "Amygdala, Attention, Dopamine, Learning, Prediction error",
author = "Roesch, {Matthew R.} and Esber, {Guillem R.} and Jian Li and Daw, {Nathaniel D.} and Geoffrey Schoenbaum",
year = "2012",
month = "4",
doi = "10.1111/j.1460-9568.2011.07986.x",
language = "English (US)",
volume = "35",
pages = "1190--1200",
journal = "European Journal of Neuroscience",
issn = "0953-816X",
publisher = "Wiley-Blackwell",
number = "7",

}

TY - JOUR

T1 - Surprise! Neural correlates of Pearce-Hall and Rescorla-Wagner coexist within the brain

AU - Roesch, Matthew R.

AU - Esber, Guillem R.

AU - Li, Jian

AU - Daw, Nathaniel D.

AU - Schoenbaum, Geoffrey

PY - 2012/4

Y1 - 2012/4

N2 - Learning theory and computational accounts suggest that learning depends on errors in outcome prediction as well as changes in processing of or attention to events. These divergent ideas are captured by models, such as Rescorla-Wagner (RW) and temporal difference (TD) learning on the one hand, which emphasize errors as directly driving changes in associative strength, vs. models such as Pearce-Hall (PH) and more recent variants on the other hand, which propose that errors promote changes in associative strength by modulating attention and processing of events. Numerous studies have shown that phasic firing of midbrain dopamine (DA) neurons carries a signed error signal consistent with RW or TD learning theories, and recently we have shown that this signal can be dissociated from attentional correlates in the basolateral amygdala and anterior cingulate. Here we will review these data along with new evidence: (i) implicating habenula and striatal regions in supporting error signaling in midbrain DA neurons; and (ii) suggesting that the central nucleus of the amygdala and prefrontal regions process the amygdalar attentional signal. However, while the neural instantiations of the RW and PH signals are dissociable and complementary, they may be linked. Any linkage would have implications for understanding why one signal dominates learning in some situations and not others, and also for appreciating the potential impact on learning of neuropathological conditions involving altered DA or amygdalar function, such as schizophrenia, addiction or anxiety disorders. Published 2012. This article is a U.S. Government work and is in the public domain in the USA.

AB - Learning theory and computational accounts suggest that learning depends on errors in outcome prediction as well as changes in processing of or attention to events. These divergent ideas are captured by models, such as Rescorla-Wagner (RW) and temporal difference (TD) learning on the one hand, which emphasize errors as directly driving changes in associative strength, vs. models such as Pearce-Hall (PH) and more recent variants on the other hand, which propose that errors promote changes in associative strength by modulating attention and processing of events. Numerous studies have shown that phasic firing of midbrain dopamine (DA) neurons carries a signed error signal consistent with RW or TD learning theories, and recently we have shown that this signal can be dissociated from attentional correlates in the basolateral amygdala and anterior cingulate. Here we will review these data along with new evidence: (i) implicating habenula and striatal regions in supporting error signaling in midbrain DA neurons; and (ii) suggesting that the central nucleus of the amygdala and prefrontal regions process the amygdalar attentional signal. However, while the neural instantiations of the RW and PH signals are dissociable and complementary, they may be linked. Any linkage would have implications for understanding why one signal dominates learning in some situations and not others, and also for appreciating the potential impact on learning of neuropathological conditions involving altered DA or amygdalar function, such as schizophrenia, addiction or anxiety disorders. Published 2012. This article is a U.S. Government work and is in the public domain in the USA.

KW - Amygdala

KW - Attention

KW - Dopamine

KW - Learning

KW - Prediction error

UR - http://www.scopus.com/inward/record.url?scp=84859339780&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84859339780&partnerID=8YFLogxK

U2 - 10.1111/j.1460-9568.2011.07986.x

DO - 10.1111/j.1460-9568.2011.07986.x

M3 - Article

C2 - 22487047

AN - SCOPUS:84859339780

VL - 35

SP - 1190

EP - 1200

JO - European Journal of Neuroscience

JF - European Journal of Neuroscience

SN - 0953-816X

IS - 7

ER -