Reinforcement learning across development: What insights can we draw from a decade of research?

Kate Nussenbaum, Catherine A. Hartley

Research output: Contribution to journalReview article

Abstract

The past decade has seen the emergence of the use of reinforcement learning models to study developmental change in value-based learning. It is unclear, however, whether these computational modeling studies, which have employed a wide variety of tasks and model variants, have reached convergent conclusions. In this review, we examine whether the tuning of model parameters that govern different aspects of learning and decision-making processes vary consistently as a function of age, and what neurocognitive developmental changes may account for differences in these parameter estimates across development. We explore whether patterns of developmental change in these estimates are better described by differences in the extent to which individuals adapt their learning processes to the statistics of different environments, or by more static learning biases that emerge across varied contexts. We focus specifically on learning rates and inverse temperature parameter estimates, and find evidence that from childhood to adulthood, individuals become better at optimally weighting recent outcomes during learning across diverse contexts and less exploratory in their value-based decision-making. We provide recommendations for how these two possibilities — and potential alternative accounts — can be tested more directly to build a cohesive body of research that yields greater insight into the development of core learning processes.

Original languageEnglish (US)
Article number100733
JournalDevelopmental Cognitive Neuroscience
Volume40
DOIs
StatePublished - Dec 2019

Fingerprint

Learning
Research
Decision Making
Reinforcement (Psychology)
Temperature

Keywords

  • Computational modeling
  • Decision making
  • Reinforcement learning

ASJC Scopus subject areas

  • Cognitive Neuroscience

Cite this

Reinforcement learning across development : What insights can we draw from a decade of research? / Nussenbaum, Kate; Hartley, Catherine A.

In: Developmental Cognitive Neuroscience, Vol. 40, 100733, 12.2019.

Research output: Contribution to journalReview article

@article{b896427a2eff47cd8dc11bb634cda2b1,
title = "Reinforcement learning across development: What insights can we draw from a decade of research?",
abstract = "The past decade has seen the emergence of the use of reinforcement learning models to study developmental change in value-based learning. It is unclear, however, whether these computational modeling studies, which have employed a wide variety of tasks and model variants, have reached convergent conclusions. In this review, we examine whether the tuning of model parameters that govern different aspects of learning and decision-making processes vary consistently as a function of age, and what neurocognitive developmental changes may account for differences in these parameter estimates across development. We explore whether patterns of developmental change in these estimates are better described by differences in the extent to which individuals adapt their learning processes to the statistics of different environments, or by more static learning biases that emerge across varied contexts. We focus specifically on learning rates and inverse temperature parameter estimates, and find evidence that from childhood to adulthood, individuals become better at optimally weighting recent outcomes during learning across diverse contexts and less exploratory in their value-based decision-making. We provide recommendations for how these two possibilities — and potential alternative accounts — can be tested more directly to build a cohesive body of research that yields greater insight into the development of core learning processes.",
keywords = "Computational modeling, Decision making, Reinforcement learning",
author = "Kate Nussenbaum and Hartley, {Catherine A.}",
year = "2019",
month = "12",
doi = "10.1016/j.dcn.2019.100733",
language = "English (US)",
volume = "40",
journal = "Developmental Cognitive Neuroscience",
issn = "1878-9293",
publisher = "Elsevier BV",

}

TY - JOUR

T1 - Reinforcement learning across development

T2 - What insights can we draw from a decade of research?

AU - Nussenbaum, Kate

AU - Hartley, Catherine A.

PY - 2019/12

Y1 - 2019/12

N2 - The past decade has seen the emergence of the use of reinforcement learning models to study developmental change in value-based learning. It is unclear, however, whether these computational modeling studies, which have employed a wide variety of tasks and model variants, have reached convergent conclusions. In this review, we examine whether the tuning of model parameters that govern different aspects of learning and decision-making processes vary consistently as a function of age, and what neurocognitive developmental changes may account for differences in these parameter estimates across development. We explore whether patterns of developmental change in these estimates are better described by differences in the extent to which individuals adapt their learning processes to the statistics of different environments, or by more static learning biases that emerge across varied contexts. We focus specifically on learning rates and inverse temperature parameter estimates, and find evidence that from childhood to adulthood, individuals become better at optimally weighting recent outcomes during learning across diverse contexts and less exploratory in their value-based decision-making. We provide recommendations for how these two possibilities — and potential alternative accounts — can be tested more directly to build a cohesive body of research that yields greater insight into the development of core learning processes.

AB - The past decade has seen the emergence of the use of reinforcement learning models to study developmental change in value-based learning. It is unclear, however, whether these computational modeling studies, which have employed a wide variety of tasks and model variants, have reached convergent conclusions. In this review, we examine whether the tuning of model parameters that govern different aspects of learning and decision-making processes vary consistently as a function of age, and what neurocognitive developmental changes may account for differences in these parameter estimates across development. We explore whether patterns of developmental change in these estimates are better described by differences in the extent to which individuals adapt their learning processes to the statistics of different environments, or by more static learning biases that emerge across varied contexts. We focus specifically on learning rates and inverse temperature parameter estimates, and find evidence that from childhood to adulthood, individuals become better at optimally weighting recent outcomes during learning across diverse contexts and less exploratory in their value-based decision-making. We provide recommendations for how these two possibilities — and potential alternative accounts — can be tested more directly to build a cohesive body of research that yields greater insight into the development of core learning processes.

KW - Computational modeling

KW - Decision making

KW - Reinforcement learning

UR - http://www.scopus.com/inward/record.url?scp=85075305939&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85075305939&partnerID=8YFLogxK

U2 - 10.1016/j.dcn.2019.100733

DO - 10.1016/j.dcn.2019.100733

M3 - Review article

AN - SCOPUS:85075305939

VL - 40

JO - Developmental Cognitive Neuroscience

JF - Developmental Cognitive Neuroscience

SN - 1878-9293

M1 - 100733

ER -