Dissociating hippocampal and striatal contributions to sequential prediction learning

Aaron M. Bornstein, Nathaniel D. Daw

Research output: Contribution to journalArticle

Abstract

Behavior may be generated on the basis of many different kinds of learned contingencies. For instance, responses could be guided by the direct association between a stimulus and response, or by sequential stimulus-stimulus relationships (as in model-based reinforcement learning or goal-directed actions). However, the neural architecture underlying sequential predictive learning is not well understood, in part because it is difficult to isolate its effect on choice behavior. To track such learning more directly, we examined reaction times (RTs) in a probabilistic sequential picture identification task in healthy individuals. We used computational learning models to isolate trial-by-trial effects of two distinct learning processes in behavior, and used these as signatures to analyse the separate neural substrates of each process. RTs were best explained via the combination of two delta rule learning processes with different learning rates. To examine neural manifestations of these learning processes, we used functional magnetic resonance imaging to seek correlates of time-series related to expectancy or surprise. We observed such correlates in two regions, hippocampus and striatum. By estimating the learning rates best explaining each signal, we verified that they were uniquely associated with one of the two distinct processes identified behaviorally. These differential correlates suggest that complementary anticipatory functions drive each region's effect on behavior. Our results provide novel insights as to the quantitative computational distinctions between medial temporal and basal ganglia learning networks and enable experiments that exploit trial-by-trial measurement of the unique contributions of both hippocampus and striatum to response behavior.

Original languageEnglish (US)
Pages (from-to)1011-1023
Number of pages13
JournalEuropean Journal of Neuroscience
Volume35
Issue number7
DOIs
StatePublished - Apr 2012

Fingerprint

Corpus Striatum
Learning
Reaction Time
Hippocampus
Choice Behavior
Basal Ganglia
Magnetic Resonance Imaging

Keywords

  • Associative learning
  • Hippocampus
  • Human
  • Model-based fMRI
  • Striatum

ASJC Scopus subject areas

  • Neuroscience(all)

Cite this

Dissociating hippocampal and striatal contributions to sequential prediction learning. / Bornstein, Aaron M.; Daw, Nathaniel D.

In: European Journal of Neuroscience, Vol. 35, No. 7, 04.2012, p. 1011-1023.

Research output: Contribution to journalArticle

Bornstein, Aaron M. ; Daw, Nathaniel D. / Dissociating hippocampal and striatal contributions to sequential prediction learning. In: European Journal of Neuroscience. 2012 ; Vol. 35, No. 7. pp. 1011-1023.
@article{aa8621892ee1402d81c51fcb5fd71633,
title = "Dissociating hippocampal and striatal contributions to sequential prediction learning",
abstract = "Behavior may be generated on the basis of many different kinds of learned contingencies. For instance, responses could be guided by the direct association between a stimulus and response, or by sequential stimulus-stimulus relationships (as in model-based reinforcement learning or goal-directed actions). However, the neural architecture underlying sequential predictive learning is not well understood, in part because it is difficult to isolate its effect on choice behavior. To track such learning more directly, we examined reaction times (RTs) in a probabilistic sequential picture identification task in healthy individuals. We used computational learning models to isolate trial-by-trial effects of two distinct learning processes in behavior, and used these as signatures to analyse the separate neural substrates of each process. RTs were best explained via the combination of two delta rule learning processes with different learning rates. To examine neural manifestations of these learning processes, we used functional magnetic resonance imaging to seek correlates of time-series related to expectancy or surprise. We observed such correlates in two regions, hippocampus and striatum. By estimating the learning rates best explaining each signal, we verified that they were uniquely associated with one of the two distinct processes identified behaviorally. These differential correlates suggest that complementary anticipatory functions drive each region's effect on behavior. Our results provide novel insights as to the quantitative computational distinctions between medial temporal and basal ganglia learning networks and enable experiments that exploit trial-by-trial measurement of the unique contributions of both hippocampus and striatum to response behavior.",
keywords = "Associative learning, Hippocampus, Human, Model-based fMRI, Striatum",
author = "Bornstein, {Aaron M.} and Daw, {Nathaniel D.}",
year = "2012",
month = "4",
doi = "10.1111/j.1460-9568.2011.07920.x",
language = "English (US)",
volume = "35",
pages = "1011--1023",
journal = "European Journal of Neuroscience",
issn = "0953-816X",
publisher = "Wiley-Blackwell",
number = "7",

}

TY - JOUR

T1 - Dissociating hippocampal and striatal contributions to sequential prediction learning

AU - Bornstein, Aaron M.

AU - Daw, Nathaniel D.

PY - 2012/4

Y1 - 2012/4

N2 - Behavior may be generated on the basis of many different kinds of learned contingencies. For instance, responses could be guided by the direct association between a stimulus and response, or by sequential stimulus-stimulus relationships (as in model-based reinforcement learning or goal-directed actions). However, the neural architecture underlying sequential predictive learning is not well understood, in part because it is difficult to isolate its effect on choice behavior. To track such learning more directly, we examined reaction times (RTs) in a probabilistic sequential picture identification task in healthy individuals. We used computational learning models to isolate trial-by-trial effects of two distinct learning processes in behavior, and used these as signatures to analyse the separate neural substrates of each process. RTs were best explained via the combination of two delta rule learning processes with different learning rates. To examine neural manifestations of these learning processes, we used functional magnetic resonance imaging to seek correlates of time-series related to expectancy or surprise. We observed such correlates in two regions, hippocampus and striatum. By estimating the learning rates best explaining each signal, we verified that they were uniquely associated with one of the two distinct processes identified behaviorally. These differential correlates suggest that complementary anticipatory functions drive each region's effect on behavior. Our results provide novel insights as to the quantitative computational distinctions between medial temporal and basal ganglia learning networks and enable experiments that exploit trial-by-trial measurement of the unique contributions of both hippocampus and striatum to response behavior.

AB - Behavior may be generated on the basis of many different kinds of learned contingencies. For instance, responses could be guided by the direct association between a stimulus and response, or by sequential stimulus-stimulus relationships (as in model-based reinforcement learning or goal-directed actions). However, the neural architecture underlying sequential predictive learning is not well understood, in part because it is difficult to isolate its effect on choice behavior. To track such learning more directly, we examined reaction times (RTs) in a probabilistic sequential picture identification task in healthy individuals. We used computational learning models to isolate trial-by-trial effects of two distinct learning processes in behavior, and used these as signatures to analyse the separate neural substrates of each process. RTs were best explained via the combination of two delta rule learning processes with different learning rates. To examine neural manifestations of these learning processes, we used functional magnetic resonance imaging to seek correlates of time-series related to expectancy or surprise. We observed such correlates in two regions, hippocampus and striatum. By estimating the learning rates best explaining each signal, we verified that they were uniquely associated with one of the two distinct processes identified behaviorally. These differential correlates suggest that complementary anticipatory functions drive each region's effect on behavior. Our results provide novel insights as to the quantitative computational distinctions between medial temporal and basal ganglia learning networks and enable experiments that exploit trial-by-trial measurement of the unique contributions of both hippocampus and striatum to response behavior.

KW - Associative learning

KW - Hippocampus

KW - Human

KW - Model-based fMRI

KW - Striatum

UR - http://www.scopus.com/inward/record.url?scp=84859297479&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84859297479&partnerID=8YFLogxK

U2 - 10.1111/j.1460-9568.2011.07920.x

DO - 10.1111/j.1460-9568.2011.07920.x

M3 - Article

VL - 35

SP - 1011

EP - 1023

JO - European Journal of Neuroscience

JF - European Journal of Neuroscience

SN - 0953-816X

IS - 7

ER -