Dynamic estimation of task-relevant variance in movement under risk

Michael S. Landy, Julia Trommershäuser, Nathaniel D. Daw

Research output: Contribution to journalArticle

Abstract

Humanstake into account theirownmovement variability as well as potential consequences of different movement outcomes in planning movement trajectories. When variability increases, planned movements are altered so as to optimize expected consequences of the movement. Past research has focused on the steady-state responses to changing conditions of movement under risk. Here, we study the dynamics of such strategy adjustment in a visuomotor decision task in which subjects reach toward a display with regions that lead to rewards and penalties, under conditions of changing uncertainty. In typical reinforcement learning tasks, subjects should base subsequent strategy by computing an estimate of the mean outcome (e.g., reward) in recent trials. In contrast, in our task, strategy should be based on a dynamic estimate of recent outcome uncertainty (i.e., squared error). We find that subjects respond to increased movement uncertainty by aiming movements more conservatively with respect to penalty regions, and that the estimate of uncertainty they use is well characterized by a weighted average of recent squared errors, with higher weights given to more recent trials.

Original languageEnglish (US)
Pages (from-to)12702-12711
Number of pages10
JournalJournal of Neuroscience
Volume32
Issue number37
DOIs
StatePublished - Sep 12 2012

Fingerprint

Uncertainty
Reward
Social Adjustment
Learning
Weights and Measures
Research

ASJC Scopus subject areas

  • Neuroscience(all)

Cite this

Dynamic estimation of task-relevant variance in movement under risk. / Landy, Michael S.; Trommershäuser, Julia; Daw, Nathaniel D.

In: Journal of Neuroscience, Vol. 32, No. 37, 12.09.2012, p. 12702-12711.

Research output: Contribution to journalArticle

Landy, Michael S. ; Trommershäuser, Julia ; Daw, Nathaniel D. / Dynamic estimation of task-relevant variance in movement under risk. In: Journal of Neuroscience. 2012 ; Vol. 32, No. 37. pp. 12702-12711.
@article{0c4e106a93d240fea2d219e1af163384,
title = "Dynamic estimation of task-relevant variance in movement under risk",
abstract = "Humanstake into account theirownmovement variability as well as potential consequences of different movement outcomes in planning movement trajectories. When variability increases, planned movements are altered so as to optimize expected consequences of the movement. Past research has focused on the steady-state responses to changing conditions of movement under risk. Here, we study the dynamics of such strategy adjustment in a visuomotor decision task in which subjects reach toward a display with regions that lead to rewards and penalties, under conditions of changing uncertainty. In typical reinforcement learning tasks, subjects should base subsequent strategy by computing an estimate of the mean outcome (e.g., reward) in recent trials. In contrast, in our task, strategy should be based on a dynamic estimate of recent outcome uncertainty (i.e., squared error). We find that subjects respond to increased movement uncertainty by aiming movements more conservatively with respect to penalty regions, and that the estimate of uncertainty they use is well characterized by a weighted average of recent squared errors, with higher weights given to more recent trials.",
author = "Landy, {Michael S.} and Julia Trommersh{\"a}user and Daw, {Nathaniel D.}",
year = "2012",
month = "9",
day = "12",
doi = "10.1523/JNEUROSCI.6160-11.2012",
language = "English (US)",
volume = "32",
pages = "12702--12711",
journal = "Journal of Neuroscience",
issn = "0270-6474",
publisher = "Society for Neuroscience",
number = "37",

}

TY - JOUR

T1 - Dynamic estimation of task-relevant variance in movement under risk

AU - Landy, Michael S.

AU - Trommershäuser, Julia

AU - Daw, Nathaniel D.

PY - 2012/9/12

Y1 - 2012/9/12

N2 - Humanstake into account theirownmovement variability as well as potential consequences of different movement outcomes in planning movement trajectories. When variability increases, planned movements are altered so as to optimize expected consequences of the movement. Past research has focused on the steady-state responses to changing conditions of movement under risk. Here, we study the dynamics of such strategy adjustment in a visuomotor decision task in which subjects reach toward a display with regions that lead to rewards and penalties, under conditions of changing uncertainty. In typical reinforcement learning tasks, subjects should base subsequent strategy by computing an estimate of the mean outcome (e.g., reward) in recent trials. In contrast, in our task, strategy should be based on a dynamic estimate of recent outcome uncertainty (i.e., squared error). We find that subjects respond to increased movement uncertainty by aiming movements more conservatively with respect to penalty regions, and that the estimate of uncertainty they use is well characterized by a weighted average of recent squared errors, with higher weights given to more recent trials.

AB - Humanstake into account theirownmovement variability as well as potential consequences of different movement outcomes in planning movement trajectories. When variability increases, planned movements are altered so as to optimize expected consequences of the movement. Past research has focused on the steady-state responses to changing conditions of movement under risk. Here, we study the dynamics of such strategy adjustment in a visuomotor decision task in which subjects reach toward a display with regions that lead to rewards and penalties, under conditions of changing uncertainty. In typical reinforcement learning tasks, subjects should base subsequent strategy by computing an estimate of the mean outcome (e.g., reward) in recent trials. In contrast, in our task, strategy should be based on a dynamic estimate of recent outcome uncertainty (i.e., squared error). We find that subjects respond to increased movement uncertainty by aiming movements more conservatively with respect to penalty regions, and that the estimate of uncertainty they use is well characterized by a weighted average of recent squared errors, with higher weights given to more recent trials.

UR - http://www.scopus.com/inward/record.url?scp=84866252910&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84866252910&partnerID=8YFLogxK

U2 - 10.1523/JNEUROSCI.6160-11.2012

DO - 10.1523/JNEUROSCI.6160-11.2012

M3 - Article

VL - 32

SP - 12702

EP - 12711

JO - Journal of Neuroscience

JF - Journal of Neuroscience

SN - 0270-6474

IS - 37

ER -