Model-based learning protects against forming habits

Claire M. Gillan, A. Ross Otto, Elizabeth A. Phelps, Nathaniel D. Daw

Research output: Contribution to journalArticle

Abstract

Studies in humans and rodents have suggested that behavior can at times be “goal-directed”—that is, planned, and purposeful—and at times “habitual”—that is, inflexible and automatically evoked by stimuli. This distinction is central to conceptions of pathological compulsion, as in drug abuse and obsessive-compulsive disorder. Evidence for the distinction has primarily come from outcome devaluation studies, in which the sensitivity of a previously learned behavior to motivational change is used to assay the dominance of habits versus goal-directed actions. However, little is known about how habits and goal-directed control arise. Specifically, in the present study we sought to reveal the trial-by-trial dynamics of instrumental learning that would promote, and protect against, developing habits. In two complementary experiments with independent samples, participants completed a sequential decision task that dissociated two computational-learning mechanisms, model-based and model-free. We then tested for habits by devaluing one of the rewards that had reinforced behavior. In each case, we found that individual differences in model-based learning predicted the participants’ subsequent sensitivity to outcome devaluation, suggesting that an associative mechanism underlies a bias toward habit formation in healthy individuals.

Original languageEnglish (US)
Pages (from-to)523-536
Number of pages14
JournalCognitive, Affective and Behavioral Neuroscience
Volume15
Issue number3
DOIs
StatePublished - Sep 8 2015

Fingerprint

Habits
Learning
Operant Conditioning
Obsessive-Compulsive Disorder
Reward
Individuality
Substance-Related Disorders
Rodentia
Outcome Assessment (Health Care)

Keywords

  • Devaluation
  • Goal-directed
  • Habit
  • Model-based
  • Model-free
  • Reinforcement learning

ASJC Scopus subject areas

  • Behavioral Neuroscience
  • Cognitive Neuroscience

Cite this

Model-based learning protects against forming habits. / Gillan, Claire M.; Otto, A. Ross; Phelps, Elizabeth A.; Daw, Nathaniel D.

In: Cognitive, Affective and Behavioral Neuroscience, Vol. 15, No. 3, 08.09.2015, p. 523-536.

Research output: Contribution to journalArticle

Gillan, Claire M. ; Otto, A. Ross ; Phelps, Elizabeth A. ; Daw, Nathaniel D. / Model-based learning protects against forming habits. In: Cognitive, Affective and Behavioral Neuroscience. 2015 ; Vol. 15, No. 3. pp. 523-536.
@article{201b4f25563c4498850cfb05e2393087,
title = "Model-based learning protects against forming habits",
abstract = "Studies in humans and rodents have suggested that behavior can at times be “goal-directed”—that is, planned, and purposeful—and at times “habitual”—that is, inflexible and automatically evoked by stimuli. This distinction is central to conceptions of pathological compulsion, as in drug abuse and obsessive-compulsive disorder. Evidence for the distinction has primarily come from outcome devaluation studies, in which the sensitivity of a previously learned behavior to motivational change is used to assay the dominance of habits versus goal-directed actions. However, little is known about how habits and goal-directed control arise. Specifically, in the present study we sought to reveal the trial-by-trial dynamics of instrumental learning that would promote, and protect against, developing habits. In two complementary experiments with independent samples, participants completed a sequential decision task that dissociated two computational-learning mechanisms, model-based and model-free. We then tested for habits by devaluing one of the rewards that had reinforced behavior. In each case, we found that individual differences in model-based learning predicted the participants’ subsequent sensitivity to outcome devaluation, suggesting that an associative mechanism underlies a bias toward habit formation in healthy individuals.",
keywords = "Devaluation, Goal-directed, Habit, Model-based, Model-free, Reinforcement learning",
author = "Gillan, {Claire M.} and Otto, {A. Ross} and Phelps, {Elizabeth A.} and Daw, {Nathaniel D.}",
year = "2015",
month = "9",
day = "8",
doi = "10.3758/s13415-015-0347-6",
language = "English (US)",
volume = "15",
pages = "523--536",
journal = "Cognitive, Affective and Behavioral Neuroscience",
issn = "1530-7026",
publisher = "Springer New York",
number = "3",

}

TY - JOUR

T1 - Model-based learning protects against forming habits

AU - Gillan, Claire M.

AU - Otto, A. Ross

AU - Phelps, Elizabeth A.

AU - Daw, Nathaniel D.

PY - 2015/9/8

Y1 - 2015/9/8

N2 - Studies in humans and rodents have suggested that behavior can at times be “goal-directed”—that is, planned, and purposeful—and at times “habitual”—that is, inflexible and automatically evoked by stimuli. This distinction is central to conceptions of pathological compulsion, as in drug abuse and obsessive-compulsive disorder. Evidence for the distinction has primarily come from outcome devaluation studies, in which the sensitivity of a previously learned behavior to motivational change is used to assay the dominance of habits versus goal-directed actions. However, little is known about how habits and goal-directed control arise. Specifically, in the present study we sought to reveal the trial-by-trial dynamics of instrumental learning that would promote, and protect against, developing habits. In two complementary experiments with independent samples, participants completed a sequential decision task that dissociated two computational-learning mechanisms, model-based and model-free. We then tested for habits by devaluing one of the rewards that had reinforced behavior. In each case, we found that individual differences in model-based learning predicted the participants’ subsequent sensitivity to outcome devaluation, suggesting that an associative mechanism underlies a bias toward habit formation in healthy individuals.

AB - Studies in humans and rodents have suggested that behavior can at times be “goal-directed”—that is, planned, and purposeful—and at times “habitual”—that is, inflexible and automatically evoked by stimuli. This distinction is central to conceptions of pathological compulsion, as in drug abuse and obsessive-compulsive disorder. Evidence for the distinction has primarily come from outcome devaluation studies, in which the sensitivity of a previously learned behavior to motivational change is used to assay the dominance of habits versus goal-directed actions. However, little is known about how habits and goal-directed control arise. Specifically, in the present study we sought to reveal the trial-by-trial dynamics of instrumental learning that would promote, and protect against, developing habits. In two complementary experiments with independent samples, participants completed a sequential decision task that dissociated two computational-learning mechanisms, model-based and model-free. We then tested for habits by devaluing one of the rewards that had reinforced behavior. In each case, we found that individual differences in model-based learning predicted the participants’ subsequent sensitivity to outcome devaluation, suggesting that an associative mechanism underlies a bias toward habit formation in healthy individuals.

KW - Devaluation

KW - Goal-directed

KW - Habit

KW - Model-based

KW - Model-free

KW - Reinforcement learning

UR - http://www.scopus.com/inward/record.url?scp=84938749786&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84938749786&partnerID=8YFLogxK

U2 - 10.3758/s13415-015-0347-6

DO - 10.3758/s13415-015-0347-6

M3 - Article

VL - 15

SP - 523

EP - 536

JO - Cognitive, Affective and Behavioral Neuroscience

JF - Cognitive, Affective and Behavioral Neuroscience

SN - 1530-7026

IS - 3

ER -