Midbrain dopamine neurons encode a quantitative reward prediction error signal

Hannah M. Bayer, Paul Glimcher

Research output: Contribution to journalArticle

Abstract

The midbrain dopamine neurons are hypothesized to provide a physiological correlate of the reward prediction error signal required by current models of reinforcement learning. We examined the activity of single dopamine neurons during a task in which subjects learned by trial and error when to make an eye movement for a juice reward. We found that these neurons encoded the difference between the current reward and a weighted average of previous rewards, a reward prediction error, but only for outcomes that were better than expected. Thus, the firing rate of midbrain dopamine neurons is quantitatively predicted by theoretical descriptions of the reward prediction error signal used in reinforcement learning models for circumstances in which this signal has a positive value. We also found that the dopamine system continued to compute the reward prediction error even when the behavioral policy of the animal was only weakly influenced by this computation.

Original languageEnglish (US)
Pages (from-to)129-141
Number of pages13
JournalNeuron
Volume47
Issue number1
DOIs
StatePublished - Jul 7 2005

Fingerprint

Dopaminergic Neurons
Mesencephalon
Reward
Learning
Eye Movements
Dopamine
Neurons

ASJC Scopus subject areas

  • Neuroscience(all)

Cite this

Midbrain dopamine neurons encode a quantitative reward prediction error signal. / Bayer, Hannah M.; Glimcher, Paul.

In: Neuron, Vol. 47, No. 1, 07.07.2005, p. 129-141.

Research output: Contribution to journalArticle

@article{5bb86ca50dfe4097bc145d448ef912af,
title = "Midbrain dopamine neurons encode a quantitative reward prediction error signal",
abstract = "The midbrain dopamine neurons are hypothesized to provide a physiological correlate of the reward prediction error signal required by current models of reinforcement learning. We examined the activity of single dopamine neurons during a task in which subjects learned by trial and error when to make an eye movement for a juice reward. We found that these neurons encoded the difference between the current reward and a weighted average of previous rewards, a reward prediction error, but only for outcomes that were better than expected. Thus, the firing rate of midbrain dopamine neurons is quantitatively predicted by theoretical descriptions of the reward prediction error signal used in reinforcement learning models for circumstances in which this signal has a positive value. We also found that the dopamine system continued to compute the reward prediction error even when the behavioral policy of the animal was only weakly influenced by this computation.",
author = "Bayer, {Hannah M.} and Paul Glimcher",
year = "2005",
month = "7",
day = "7",
doi = "10.1016/j.neuron.2005.05.020",
language = "English (US)",
volume = "47",
pages = "129--141",
journal = "Neuron",
issn = "0896-6273",
publisher = "Cell Press",
number = "1",

}

TY - JOUR

T1 - Midbrain dopamine neurons encode a quantitative reward prediction error signal

AU - Bayer, Hannah M.

AU - Glimcher, Paul

PY - 2005/7/7

Y1 - 2005/7/7

N2 - The midbrain dopamine neurons are hypothesized to provide a physiological correlate of the reward prediction error signal required by current models of reinforcement learning. We examined the activity of single dopamine neurons during a task in which subjects learned by trial and error when to make an eye movement for a juice reward. We found that these neurons encoded the difference between the current reward and a weighted average of previous rewards, a reward prediction error, but only for outcomes that were better than expected. Thus, the firing rate of midbrain dopamine neurons is quantitatively predicted by theoretical descriptions of the reward prediction error signal used in reinforcement learning models for circumstances in which this signal has a positive value. We also found that the dopamine system continued to compute the reward prediction error even when the behavioral policy of the animal was only weakly influenced by this computation.

AB - The midbrain dopamine neurons are hypothesized to provide a physiological correlate of the reward prediction error signal required by current models of reinforcement learning. We examined the activity of single dopamine neurons during a task in which subjects learned by trial and error when to make an eye movement for a juice reward. We found that these neurons encoded the difference between the current reward and a weighted average of previous rewards, a reward prediction error, but only for outcomes that were better than expected. Thus, the firing rate of midbrain dopamine neurons is quantitatively predicted by theoretical descriptions of the reward prediction error signal used in reinforcement learning models for circumstances in which this signal has a positive value. We also found that the dopamine system continued to compute the reward prediction error even when the behavioral policy of the animal was only weakly influenced by this computation.

UR - http://www.scopus.com/inward/record.url?scp=21544435722&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=21544435722&partnerID=8YFLogxK

U2 - 10.1016/j.neuron.2005.05.020

DO - 10.1016/j.neuron.2005.05.020

M3 - Article

C2 - 15996553

AN - SCOPUS:21544435722

VL - 47

SP - 129

EP - 141

JO - Neuron

JF - Neuron

SN - 0896-6273

IS - 1

ER -