Neural circuit mechanisms of value-based decision-making and reinforcement learning

A. Soltani, W. Chaisangmongkon, Xiao-Jing Wang

Research output: Chapter in Book/Report/Conference proceedingChapter

Abstract

Despite groundbreaking progress, currently we still know preciously little about the biophysical and circuit mechanisms of valuation and reward-dependent plasticity underlying adaptive choice behavior. For instance, whereas phasic firing of dopamine neurons has long been ascribed to represent reward-prediction error (RPE), only recently has research begun to uncover the mechanism of how such a signal is computed at the circuit level. In this chapter, we will briefly review neuroscience experiments and mathematical models on reward-dependent adaptive choice behavior and then focus on a biologically plausible, reward-modulated Hebbian synaptic plasticity rule. We will show that a decision-making neural circuit endowed with this learning rule is capable of accounting for behavioral and neurophysiological observations in a variety of value-based decision-making tasks, including foraging, competitive games, and probabilistic inference. Looking forward, an outstanding challenge is to elucidate the distributed nature of reward-dependent processes across a large-scale brain system.

Original languageEnglish (US)
Title of host publicationDecision Neuroscience
Subtitle of host publicationAn Integrative Perspective
PublisherElsevier Inc.
Pages233-245
Number of pages13
ISBN (Electronic)9780128053317
ISBN (Print)9780128053089
DOIs
StatePublished - Oct 10 2016

Fingerprint

Reward
Decision Making
Learning
Choice Behavior
Psychological Adaptation
Neuronal Plasticity
Dopaminergic Neurons
Neurosciences
Theoretical Models
Reinforcement (Psychology)
Brain
Research

Keywords

  • Competitive game
  • Computational principles
  • Matching law
  • Neural circuit mechanism
  • Probabilistic inference
  • Single-neuron physiology
  • Valuation computation
  • Value-based adaptive choice behavior

ASJC Scopus subject areas

  • Medicine(all)
  • Neuroscience(all)

Cite this

Soltani, A., Chaisangmongkon, W., & Wang, X-J. (2016). Neural circuit mechanisms of value-based decision-making and reinforcement learning. In Decision Neuroscience: An Integrative Perspective (pp. 233-245). Elsevier Inc.. https://doi.org/10.1016/B978-0-12-805308-9.00013-0

Neural circuit mechanisms of value-based decision-making and reinforcement learning. / Soltani, A.; Chaisangmongkon, W.; Wang, Xiao-Jing.

Decision Neuroscience: An Integrative Perspective. Elsevier Inc., 2016. p. 233-245.

Research output: Chapter in Book/Report/Conference proceedingChapter

Soltani, A, Chaisangmongkon, W & Wang, X-J 2016, Neural circuit mechanisms of value-based decision-making and reinforcement learning. in Decision Neuroscience: An Integrative Perspective. Elsevier Inc., pp. 233-245. https://doi.org/10.1016/B978-0-12-805308-9.00013-0
Soltani A, Chaisangmongkon W, Wang X-J. Neural circuit mechanisms of value-based decision-making and reinforcement learning. In Decision Neuroscience: An Integrative Perspective. Elsevier Inc. 2016. p. 233-245 https://doi.org/10.1016/B978-0-12-805308-9.00013-0
Soltani, A. ; Chaisangmongkon, W. ; Wang, Xiao-Jing. / Neural circuit mechanisms of value-based decision-making and reinforcement learning. Decision Neuroscience: An Integrative Perspective. Elsevier Inc., 2016. pp. 233-245
@inbook{9e0a98ff71524ecface051ec4dae1fb6,
title = "Neural circuit mechanisms of value-based decision-making and reinforcement learning",
abstract = "Despite groundbreaking progress, currently we still know preciously little about the biophysical and circuit mechanisms of valuation and reward-dependent plasticity underlying adaptive choice behavior. For instance, whereas phasic firing of dopamine neurons has long been ascribed to represent reward-prediction error (RPE), only recently has research begun to uncover the mechanism of how such a signal is computed at the circuit level. In this chapter, we will briefly review neuroscience experiments and mathematical models on reward-dependent adaptive choice behavior and then focus on a biologically plausible, reward-modulated Hebbian synaptic plasticity rule. We will show that a decision-making neural circuit endowed with this learning rule is capable of accounting for behavioral and neurophysiological observations in a variety of value-based decision-making tasks, including foraging, competitive games, and probabilistic inference. Looking forward, an outstanding challenge is to elucidate the distributed nature of reward-dependent processes across a large-scale brain system.",
keywords = "Competitive game, Computational principles, Matching law, Neural circuit mechanism, Probabilistic inference, Single-neuron physiology, Valuation computation, Value-based adaptive choice behavior",
author = "A. Soltani and W. Chaisangmongkon and Xiao-Jing Wang",
year = "2016",
month = "10",
day = "10",
doi = "10.1016/B978-0-12-805308-9.00013-0",
language = "English (US)",
isbn = "9780128053089",
pages = "233--245",
booktitle = "Decision Neuroscience",
publisher = "Elsevier Inc.",
address = "United States",

}

TY - CHAP

T1 - Neural circuit mechanisms of value-based decision-making and reinforcement learning

AU - Soltani, A.

AU - Chaisangmongkon, W.

AU - Wang, Xiao-Jing

PY - 2016/10/10

Y1 - 2016/10/10

N2 - Despite groundbreaking progress, currently we still know preciously little about the biophysical and circuit mechanisms of valuation and reward-dependent plasticity underlying adaptive choice behavior. For instance, whereas phasic firing of dopamine neurons has long been ascribed to represent reward-prediction error (RPE), only recently has research begun to uncover the mechanism of how such a signal is computed at the circuit level. In this chapter, we will briefly review neuroscience experiments and mathematical models on reward-dependent adaptive choice behavior and then focus on a biologically plausible, reward-modulated Hebbian synaptic plasticity rule. We will show that a decision-making neural circuit endowed with this learning rule is capable of accounting for behavioral and neurophysiological observations in a variety of value-based decision-making tasks, including foraging, competitive games, and probabilistic inference. Looking forward, an outstanding challenge is to elucidate the distributed nature of reward-dependent processes across a large-scale brain system.

AB - Despite groundbreaking progress, currently we still know preciously little about the biophysical and circuit mechanisms of valuation and reward-dependent plasticity underlying adaptive choice behavior. For instance, whereas phasic firing of dopamine neurons has long been ascribed to represent reward-prediction error (RPE), only recently has research begun to uncover the mechanism of how such a signal is computed at the circuit level. In this chapter, we will briefly review neuroscience experiments and mathematical models on reward-dependent adaptive choice behavior and then focus on a biologically plausible, reward-modulated Hebbian synaptic plasticity rule. We will show that a decision-making neural circuit endowed with this learning rule is capable of accounting for behavioral and neurophysiological observations in a variety of value-based decision-making tasks, including foraging, competitive games, and probabilistic inference. Looking forward, an outstanding challenge is to elucidate the distributed nature of reward-dependent processes across a large-scale brain system.

KW - Competitive game

KW - Computational principles

KW - Matching law

KW - Neural circuit mechanism

KW - Probabilistic inference

KW - Single-neuron physiology

KW - Valuation computation

KW - Value-based adaptive choice behavior

UR - http://www.scopus.com/inward/record.url?scp=85020511215&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85020511215&partnerID=8YFLogxK

U2 - 10.1016/B978-0-12-805308-9.00013-0

DO - 10.1016/B978-0-12-805308-9.00013-0

M3 - Chapter

SN - 9780128053089

SP - 233

EP - 245

BT - Decision Neuroscience

PB - Elsevier Inc.

ER -