Neural mechanism for stochastic behaviour during a competitive game

Alireza Soltani, Daeyeol Lee, Xiao-Jing Wang

Research output: Contribution to journalArticle

Abstract

Previous studies have shown that non-human primates can generate highly stochastic choice behaviour, especially when this is required during a competitive interaction with another agent. To understand the neural mechanism of such dynamic choice behaviour, we propose a biologically plausible model of decision making endowed with synaptic plasticity that follows a reward-dependent stochastic Hebbian learning rule. This model constitutes a biophysical implementation of reinforcement learning, and it reproduces salient features of behavioural data from an experiment with monkeys playing a matching pennies game. Due to interaction with an opponent and learning dynamics, the model generates quasi-random behaviour robustly in spite of intrinsic biases. Furthermore, non-random choice behaviour can also emerge when the model plays against a non-interactive opponent, as observed in the monkey experiment. Finally, when combined with a meta-learning algorithm, our model accounts for the slow drift in the animal's strategy based on a process of reward maximization.

Original languageEnglish (US)
Pages (from-to)1075-1090
Number of pages16
JournalNeural Networks
Volume19
Issue number8
DOIs
StatePublished - Oct 2006

Fingerprint

Choice Behavior
Learning
Reward
Haplorhini
Neuronal Plasticity
Primates
Reinforcement learning
Decision Making
Learning algorithms
Plasticity
Animals
Decision making
Experiments

Keywords

  • Decision making
  • Game theory
  • Meta-learning
  • Reinforcement learning
  • Reward-dependent stochastic Hebbian learning rule
  • Synaptic plasticity

ASJC Scopus subject areas

  • Artificial Intelligence
  • Neuroscience(all)

Cite this

Neural mechanism for stochastic behaviour during a competitive game. / Soltani, Alireza; Lee, Daeyeol; Wang, Xiao-Jing.

In: Neural Networks, Vol. 19, No. 8, 10.2006, p. 1075-1090.

Research output: Contribution to journalArticle

Soltani, Alireza ; Lee, Daeyeol ; Wang, Xiao-Jing. / Neural mechanism for stochastic behaviour during a competitive game. In: Neural Networks. 2006 ; Vol. 19, No. 8. pp. 1075-1090.
@article{4f37e173d2444581a0d2417f2f188d7e,
title = "Neural mechanism for stochastic behaviour during a competitive game",
abstract = "Previous studies have shown that non-human primates can generate highly stochastic choice behaviour, especially when this is required during a competitive interaction with another agent. To understand the neural mechanism of such dynamic choice behaviour, we propose a biologically plausible model of decision making endowed with synaptic plasticity that follows a reward-dependent stochastic Hebbian learning rule. This model constitutes a biophysical implementation of reinforcement learning, and it reproduces salient features of behavioural data from an experiment with monkeys playing a matching pennies game. Due to interaction with an opponent and learning dynamics, the model generates quasi-random behaviour robustly in spite of intrinsic biases. Furthermore, non-random choice behaviour can also emerge when the model plays against a non-interactive opponent, as observed in the monkey experiment. Finally, when combined with a meta-learning algorithm, our model accounts for the slow drift in the animal's strategy based on a process of reward maximization.",
keywords = "Decision making, Game theory, Meta-learning, Reinforcement learning, Reward-dependent stochastic Hebbian learning rule, Synaptic plasticity",
author = "Alireza Soltani and Daeyeol Lee and Xiao-Jing Wang",
year = "2006",
month = "10",
doi = "10.1016/j.neunet.2006.05.044",
language = "English (US)",
volume = "19",
pages = "1075--1090",
journal = "Neural Networks",
issn = "0893-6080",
publisher = "Elsevier Limited",
number = "8",

}

TY - JOUR

T1 - Neural mechanism for stochastic behaviour during a competitive game

AU - Soltani, Alireza

AU - Lee, Daeyeol

AU - Wang, Xiao-Jing

PY - 2006/10

Y1 - 2006/10

N2 - Previous studies have shown that non-human primates can generate highly stochastic choice behaviour, especially when this is required during a competitive interaction with another agent. To understand the neural mechanism of such dynamic choice behaviour, we propose a biologically plausible model of decision making endowed with synaptic plasticity that follows a reward-dependent stochastic Hebbian learning rule. This model constitutes a biophysical implementation of reinforcement learning, and it reproduces salient features of behavioural data from an experiment with monkeys playing a matching pennies game. Due to interaction with an opponent and learning dynamics, the model generates quasi-random behaviour robustly in spite of intrinsic biases. Furthermore, non-random choice behaviour can also emerge when the model plays against a non-interactive opponent, as observed in the monkey experiment. Finally, when combined with a meta-learning algorithm, our model accounts for the slow drift in the animal's strategy based on a process of reward maximization.

AB - Previous studies have shown that non-human primates can generate highly stochastic choice behaviour, especially when this is required during a competitive interaction with another agent. To understand the neural mechanism of such dynamic choice behaviour, we propose a biologically plausible model of decision making endowed with synaptic plasticity that follows a reward-dependent stochastic Hebbian learning rule. This model constitutes a biophysical implementation of reinforcement learning, and it reproduces salient features of behavioural data from an experiment with monkeys playing a matching pennies game. Due to interaction with an opponent and learning dynamics, the model generates quasi-random behaviour robustly in spite of intrinsic biases. Furthermore, non-random choice behaviour can also emerge when the model plays against a non-interactive opponent, as observed in the monkey experiment. Finally, when combined with a meta-learning algorithm, our model accounts for the slow drift in the animal's strategy based on a process of reward maximization.

KW - Decision making

KW - Game theory

KW - Meta-learning

KW - Reinforcement learning

KW - Reward-dependent stochastic Hebbian learning rule

KW - Synaptic plasticity

UR - http://www.scopus.com/inward/record.url?scp=33748999594&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33748999594&partnerID=8YFLogxK

U2 - 10.1016/j.neunet.2006.05.044

DO - 10.1016/j.neunet.2006.05.044

M3 - Article

C2 - 17015181

AN - SCOPUS:33748999594

VL - 19

SP - 1075

EP - 1090

JO - Neural Networks

JF - Neural Networks

SN - 0893-6080

IS - 8

ER -