A biophysically based neural model of matching law behavior

Melioration by stochastic synapses

Alireza Soltani, Xiao-Jing Wang

Research output: Contribution to journalArticle

Abstract

In experiments designed to uncover the neural basis of adaptive decision making in a foraging environment, neuroscientists have reported single-cell activities in the lateral intraparietal cortex (LIP) that are correlated with choice options and their subjective values. To investigate the underlying synaptic mechanism, we considered a spiking neuron model of decision making endowed with synaptic plasticity that follows a reward-dependent stochastic Hebbian learning rule. This general model is tested in a matching task in which rewards on two targets are scheduled randomly with different rates. Our main results are threefold. First, we show that plastic synapses provide a natural way to integrate past rewards and estimate the local (in time) "return" of a choice. Second, our model reproduces the matching behavior (i.e., the proportional allocation of choices matches the relative reinforcement obtained on those choices, which is achieved through melioration in individual trials). Our model also explains the observed " undermatching" phenomenon and points to biophysical constraints (such as finite learning rate and stochastic neuronal firing) that set the limits to matching behavior. Third, although our decision model is an attractor network exhibiting winner-take-all competition, it captures graded neural spiking activities observed in LIP, when the latter were sorted according to the choices and the difference in the returns for the two targets. These results suggest that neurons in LIP are involved in selecting the oculomotor responses, whereas rewards are integrated and stored elsewhere, possibly by plastic synapses and in the form of the return rather than income of choice options.

Original languageEnglish (US)
Pages (from-to)3731-3744
Number of pages14
JournalJournal of Neuroscience
Volume26
Issue number14
DOIs
StatePublished - Apr 5 2006

Fingerprint

Reward
Synapses
Plastics
Decision Making
Learning
Neurons
Neuronal Plasticity

Keywords

  • Decision making
  • Dopamine
  • Lateral intraparietal cortex
  • Matching behavior
  • Melioration
  • Reward-dependent stochastic Hebbian learning

ASJC Scopus subject areas

  • Neuroscience(all)

Cite this

A biophysically based neural model of matching law behavior : Melioration by stochastic synapses. / Soltani, Alireza; Wang, Xiao-Jing.

In: Journal of Neuroscience, Vol. 26, No. 14, 05.04.2006, p. 3731-3744.

Research output: Contribution to journalArticle

@article{c46f5f35d64a41edba6137b0e369ea62,
title = "A biophysically based neural model of matching law behavior: Melioration by stochastic synapses",
abstract = "In experiments designed to uncover the neural basis of adaptive decision making in a foraging environment, neuroscientists have reported single-cell activities in the lateral intraparietal cortex (LIP) that are correlated with choice options and their subjective values. To investigate the underlying synaptic mechanism, we considered a spiking neuron model of decision making endowed with synaptic plasticity that follows a reward-dependent stochastic Hebbian learning rule. This general model is tested in a matching task in which rewards on two targets are scheduled randomly with different rates. Our main results are threefold. First, we show that plastic synapses provide a natural way to integrate past rewards and estimate the local (in time) {"}return{"} of a choice. Second, our model reproduces the matching behavior (i.e., the proportional allocation of choices matches the relative reinforcement obtained on those choices, which is achieved through melioration in individual trials). Our model also explains the observed {"} undermatching{"} phenomenon and points to biophysical constraints (such as finite learning rate and stochastic neuronal firing) that set the limits to matching behavior. Third, although our decision model is an attractor network exhibiting winner-take-all competition, it captures graded neural spiking activities observed in LIP, when the latter were sorted according to the choices and the difference in the returns for the two targets. These results suggest that neurons in LIP are involved in selecting the oculomotor responses, whereas rewards are integrated and stored elsewhere, possibly by plastic synapses and in the form of the return rather than income of choice options.",
keywords = "Decision making, Dopamine, Lateral intraparietal cortex, Matching behavior, Melioration, Reward-dependent stochastic Hebbian learning",
author = "Alireza Soltani and Xiao-Jing Wang",
year = "2006",
month = "4",
day = "5",
doi = "10.1523/JNEUROSCI.5159-05.2006",
language = "English (US)",
volume = "26",
pages = "3731--3744",
journal = "Journal of Neuroscience",
issn = "0270-6474",
publisher = "Society for Neuroscience",
number = "14",

}

TY - JOUR

T1 - A biophysically based neural model of matching law behavior

T2 - Melioration by stochastic synapses

AU - Soltani, Alireza

AU - Wang, Xiao-Jing

PY - 2006/4/5

Y1 - 2006/4/5

N2 - In experiments designed to uncover the neural basis of adaptive decision making in a foraging environment, neuroscientists have reported single-cell activities in the lateral intraparietal cortex (LIP) that are correlated with choice options and their subjective values. To investigate the underlying synaptic mechanism, we considered a spiking neuron model of decision making endowed with synaptic plasticity that follows a reward-dependent stochastic Hebbian learning rule. This general model is tested in a matching task in which rewards on two targets are scheduled randomly with different rates. Our main results are threefold. First, we show that plastic synapses provide a natural way to integrate past rewards and estimate the local (in time) "return" of a choice. Second, our model reproduces the matching behavior (i.e., the proportional allocation of choices matches the relative reinforcement obtained on those choices, which is achieved through melioration in individual trials). Our model also explains the observed " undermatching" phenomenon and points to biophysical constraints (such as finite learning rate and stochastic neuronal firing) that set the limits to matching behavior. Third, although our decision model is an attractor network exhibiting winner-take-all competition, it captures graded neural spiking activities observed in LIP, when the latter were sorted according to the choices and the difference in the returns for the two targets. These results suggest that neurons in LIP are involved in selecting the oculomotor responses, whereas rewards are integrated and stored elsewhere, possibly by plastic synapses and in the form of the return rather than income of choice options.

AB - In experiments designed to uncover the neural basis of adaptive decision making in a foraging environment, neuroscientists have reported single-cell activities in the lateral intraparietal cortex (LIP) that are correlated with choice options and their subjective values. To investigate the underlying synaptic mechanism, we considered a spiking neuron model of decision making endowed with synaptic plasticity that follows a reward-dependent stochastic Hebbian learning rule. This general model is tested in a matching task in which rewards on two targets are scheduled randomly with different rates. Our main results are threefold. First, we show that plastic synapses provide a natural way to integrate past rewards and estimate the local (in time) "return" of a choice. Second, our model reproduces the matching behavior (i.e., the proportional allocation of choices matches the relative reinforcement obtained on those choices, which is achieved through melioration in individual trials). Our model also explains the observed " undermatching" phenomenon and points to biophysical constraints (such as finite learning rate and stochastic neuronal firing) that set the limits to matching behavior. Third, although our decision model is an attractor network exhibiting winner-take-all competition, it captures graded neural spiking activities observed in LIP, when the latter were sorted according to the choices and the difference in the returns for the two targets. These results suggest that neurons in LIP are involved in selecting the oculomotor responses, whereas rewards are integrated and stored elsewhere, possibly by plastic synapses and in the form of the return rather than income of choice options.

KW - Decision making

KW - Dopamine

KW - Lateral intraparietal cortex

KW - Matching behavior

KW - Melioration

KW - Reward-dependent stochastic Hebbian learning

UR - http://www.scopus.com/inward/record.url?scp=33645566919&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33645566919&partnerID=8YFLogxK

U2 - 10.1523/JNEUROSCI.5159-05.2006

DO - 10.1523/JNEUROSCI.5159-05.2006

M3 - Article

VL - 26

SP - 3731

EP - 3744

JO - Journal of Neuroscience

JF - Journal of Neuroscience

SN - 0270-6474

IS - 14

ER -