From Creatures of Habit to Goal-Directed Learners: Tracking the Developmental Emergence of Model-Based Reinforcement Learning

Johannes H. Decker, A. Ross Otto, Nathaniel D. Daw, Catherine Hartley

Research output: Contribution to journalArticle

Abstract

Theoretical models distinguish two decision-making strategies that have been formalized in reinforcement-learning theory. A model-based strategy leverages a cognitive model of potential actions and their consequences to make goal-directed choices, whereas a model-free strategy evaluates actions based solely on their reward history. Research in adults has begun to elucidate the psychological mechanisms and neural substrates underlying these learning processes and factors that influence their relative recruitment. However, the developmental trajectory of these evaluative strategies has not been well characterized. In this study, children, adolescents, and adults performed a sequential reinforcement-learning task that enabled estimation of model-based and model-free contributions to choice. Whereas a model-free strategy was apparent in choice behavior across all age groups, a model-based strategy was absent in children, became evident in adolescents, and strengthened in adults. These results suggest that recruitment of model-based valuation systems represents a critical cognitive component underlying the gradual maturation of goal-directed behavior.

Original languageEnglish (US)
Pages (from-to)848-858
Number of pages11
JournalPsychological Science
Volume27
Issue number6
DOIs
StatePublished - Jun 1 2016

Fingerprint

Habits
Learning
Choice Behavior
Reward
Action Potentials
Decision Making
Theoretical Models
Age Groups
History
Psychology
Research
Reinforcement (Psychology)
Habit
Creatures
Reinforcement Learning

Keywords

  • cognitive development
  • decision making
  • open data
  • reinforcement learning

ASJC Scopus subject areas

  • Arts and Humanities (miscellaneous)
  • Psychology(all)

Cite this

From Creatures of Habit to Goal-Directed Learners : Tracking the Developmental Emergence of Model-Based Reinforcement Learning. / Decker, Johannes H.; Otto, A. Ross; Daw, Nathaniel D.; Hartley, Catherine.

In: Psychological Science, Vol. 27, No. 6, 01.06.2016, p. 848-858.

Research output: Contribution to journalArticle

@article{f4d7f463b4b8422ab306a6a718c01364,
title = "From Creatures of Habit to Goal-Directed Learners: Tracking the Developmental Emergence of Model-Based Reinforcement Learning",
abstract = "Theoretical models distinguish two decision-making strategies that have been formalized in reinforcement-learning theory. A model-based strategy leverages a cognitive model of potential actions and their consequences to make goal-directed choices, whereas a model-free strategy evaluates actions based solely on their reward history. Research in adults has begun to elucidate the psychological mechanisms and neural substrates underlying these learning processes and factors that influence their relative recruitment. However, the developmental trajectory of these evaluative strategies has not been well characterized. In this study, children, adolescents, and adults performed a sequential reinforcement-learning task that enabled estimation of model-based and model-free contributions to choice. Whereas a model-free strategy was apparent in choice behavior across all age groups, a model-based strategy was absent in children, became evident in adolescents, and strengthened in adults. These results suggest that recruitment of model-based valuation systems represents a critical cognitive component underlying the gradual maturation of goal-directed behavior.",
keywords = "cognitive development, decision making, open data, reinforcement learning",
author = "Decker, {Johannes H.} and Otto, {A. Ross} and Daw, {Nathaniel D.} and Catherine Hartley",
year = "2016",
month = "6",
day = "1",
doi = "10.1177/0956797616639301",
language = "English (US)",
volume = "27",
pages = "848--858",
journal = "Psychological Science",
issn = "0956-7976",
publisher = "SAGE Publications Inc.",
number = "6",

}

TY - JOUR

T1 - From Creatures of Habit to Goal-Directed Learners

T2 - Tracking the Developmental Emergence of Model-Based Reinforcement Learning

AU - Decker, Johannes H.

AU - Otto, A. Ross

AU - Daw, Nathaniel D.

AU - Hartley, Catherine

PY - 2016/6/1

Y1 - 2016/6/1

N2 - Theoretical models distinguish two decision-making strategies that have been formalized in reinforcement-learning theory. A model-based strategy leverages a cognitive model of potential actions and their consequences to make goal-directed choices, whereas a model-free strategy evaluates actions based solely on their reward history. Research in adults has begun to elucidate the psychological mechanisms and neural substrates underlying these learning processes and factors that influence their relative recruitment. However, the developmental trajectory of these evaluative strategies has not been well characterized. In this study, children, adolescents, and adults performed a sequential reinforcement-learning task that enabled estimation of model-based and model-free contributions to choice. Whereas a model-free strategy was apparent in choice behavior across all age groups, a model-based strategy was absent in children, became evident in adolescents, and strengthened in adults. These results suggest that recruitment of model-based valuation systems represents a critical cognitive component underlying the gradual maturation of goal-directed behavior.

AB - Theoretical models distinguish two decision-making strategies that have been formalized in reinforcement-learning theory. A model-based strategy leverages a cognitive model of potential actions and their consequences to make goal-directed choices, whereas a model-free strategy evaluates actions based solely on their reward history. Research in adults has begun to elucidate the psychological mechanisms and neural substrates underlying these learning processes and factors that influence their relative recruitment. However, the developmental trajectory of these evaluative strategies has not been well characterized. In this study, children, adolescents, and adults performed a sequential reinforcement-learning task that enabled estimation of model-based and model-free contributions to choice. Whereas a model-free strategy was apparent in choice behavior across all age groups, a model-based strategy was absent in children, became evident in adolescents, and strengthened in adults. These results suggest that recruitment of model-based valuation systems represents a critical cognitive component underlying the gradual maturation of goal-directed behavior.

KW - cognitive development

KW - decision making

KW - open data

KW - reinforcement learning

UR - http://www.scopus.com/inward/record.url?scp=84983050082&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84983050082&partnerID=8YFLogxK

U2 - 10.1177/0956797616639301

DO - 10.1177/0956797616639301

M3 - Article

C2 - 27084852

AN - SCOPUS:84983050082

VL - 27

SP - 848

EP - 858

JO - Psychological Science

JF - Psychological Science

SN - 0956-7976

IS - 6

ER -