Dynamic response-by-response models of matching behavior in rhesus monkeys

Brian Lau, Paul Glimcher

Research output: Contribution to journalArticle

Abstract

We studied the choice behavior of 2 monkeys in a discrete-trial task with reinforcement contingencies similar to those Herrnstein (1961) used when he described the matching law. In each session, the monkeys experienced blocks of discrete trials at different relative-reinforcer frequencies or magnitudes with unsignalled transitions between the blocks. Steady-state data following adjustment to each transition were well characterized by the generalized matching law; response ratios undermatched reinforcer frequency ratios but matched reinforcer magnitude ratios. We modelled response-by-response behavior with linear models that used past reinforcers as well as past choices to predict the monkeys' choices on each trial. We found that more recently obtained reinforcers more strongly influenced choice behavior. Perhaps surprisingly, we also found that the monkeys' actions were influenced by the pattern of their own past choices. It was necessary to incorporate both past reinforcers and past choices in order to accurately capture steady-state behavior as well as the fluctuations during block transitions and the response-by-response patterns of behavior. Our results suggest that simple reinforcement learning models must account for the effects of past choices to accurately characterize behavior in this task, and that models with these properties provide a conceptual tool for studying how both past reinforcers and past choices are integrated by the neural systems that generate behavior.

Original languageEnglish (US)
Pages (from-to)555-579
Number of pages25
JournalJournal of the Experimental Analysis of Behavior
Volume84
Issue number3
DOIs
StatePublished - Nov 2005

Fingerprint

Macaca mulatta
Haplorhini
Choice Behavior
Linear Models
Research Design
Learning
Reinforcement (Psychology)

Keywords

  • Choice
  • Dynamics
  • Eye movement
  • Matching law
  • Model
  • Monkey

ASJC Scopus subject areas

  • Behavioral Neuroscience
  • Psychology(all)
  • Neuropsychology and Physiological Psychology
  • Experimental and Cognitive Psychology

Cite this

Dynamic response-by-response models of matching behavior in rhesus monkeys. / Lau, Brian; Glimcher, Paul.

In: Journal of the Experimental Analysis of Behavior, Vol. 84, No. 3, 11.2005, p. 555-579.

Research output: Contribution to journalArticle

@article{9ada9e6fba5a47c4a63bf388e43d528c,
title = "Dynamic response-by-response models of matching behavior in rhesus monkeys",
abstract = "We studied the choice behavior of 2 monkeys in a discrete-trial task with reinforcement contingencies similar to those Herrnstein (1961) used when he described the matching law. In each session, the monkeys experienced blocks of discrete trials at different relative-reinforcer frequencies or magnitudes with unsignalled transitions between the blocks. Steady-state data following adjustment to each transition were well characterized by the generalized matching law; response ratios undermatched reinforcer frequency ratios but matched reinforcer magnitude ratios. We modelled response-by-response behavior with linear models that used past reinforcers as well as past choices to predict the monkeys' choices on each trial. We found that more recently obtained reinforcers more strongly influenced choice behavior. Perhaps surprisingly, we also found that the monkeys' actions were influenced by the pattern of their own past choices. It was necessary to incorporate both past reinforcers and past choices in order to accurately capture steady-state behavior as well as the fluctuations during block transitions and the response-by-response patterns of behavior. Our results suggest that simple reinforcement learning models must account for the effects of past choices to accurately characterize behavior in this task, and that models with these properties provide a conceptual tool for studying how both past reinforcers and past choices are integrated by the neural systems that generate behavior.",
keywords = "Choice, Dynamics, Eye movement, Matching law, Model, Monkey",
author = "Brian Lau and Paul Glimcher",
year = "2005",
month = "11",
doi = "10.1901/jeab.2005.110-04",
language = "English (US)",
volume = "84",
pages = "555--579",
journal = "Journal of the Experimental Analysis of Behavior",
issn = "0022-5002",
publisher = "Society for the Experimental Analysis of Behavior Inc.",
number = "3",

}

TY - JOUR

T1 - Dynamic response-by-response models of matching behavior in rhesus monkeys

AU - Lau, Brian

AU - Glimcher, Paul

PY - 2005/11

Y1 - 2005/11

N2 - We studied the choice behavior of 2 monkeys in a discrete-trial task with reinforcement contingencies similar to those Herrnstein (1961) used when he described the matching law. In each session, the monkeys experienced blocks of discrete trials at different relative-reinforcer frequencies or magnitudes with unsignalled transitions between the blocks. Steady-state data following adjustment to each transition were well characterized by the generalized matching law; response ratios undermatched reinforcer frequency ratios but matched reinforcer magnitude ratios. We modelled response-by-response behavior with linear models that used past reinforcers as well as past choices to predict the monkeys' choices on each trial. We found that more recently obtained reinforcers more strongly influenced choice behavior. Perhaps surprisingly, we also found that the monkeys' actions were influenced by the pattern of their own past choices. It was necessary to incorporate both past reinforcers and past choices in order to accurately capture steady-state behavior as well as the fluctuations during block transitions and the response-by-response patterns of behavior. Our results suggest that simple reinforcement learning models must account for the effects of past choices to accurately characterize behavior in this task, and that models with these properties provide a conceptual tool for studying how both past reinforcers and past choices are integrated by the neural systems that generate behavior.

AB - We studied the choice behavior of 2 monkeys in a discrete-trial task with reinforcement contingencies similar to those Herrnstein (1961) used when he described the matching law. In each session, the monkeys experienced blocks of discrete trials at different relative-reinforcer frequencies or magnitudes with unsignalled transitions between the blocks. Steady-state data following adjustment to each transition were well characterized by the generalized matching law; response ratios undermatched reinforcer frequency ratios but matched reinforcer magnitude ratios. We modelled response-by-response behavior with linear models that used past reinforcers as well as past choices to predict the monkeys' choices on each trial. We found that more recently obtained reinforcers more strongly influenced choice behavior. Perhaps surprisingly, we also found that the monkeys' actions were influenced by the pattern of their own past choices. It was necessary to incorporate both past reinforcers and past choices in order to accurately capture steady-state behavior as well as the fluctuations during block transitions and the response-by-response patterns of behavior. Our results suggest that simple reinforcement learning models must account for the effects of past choices to accurately characterize behavior in this task, and that models with these properties provide a conceptual tool for studying how both past reinforcers and past choices are integrated by the neural systems that generate behavior.

KW - Choice

KW - Dynamics

KW - Eye movement

KW - Matching law

KW - Model

KW - Monkey

UR - http://www.scopus.com/inward/record.url?scp=33644782012&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33644782012&partnerID=8YFLogxK

U2 - 10.1901/jeab.2005.110-04

DO - 10.1901/jeab.2005.110-04

M3 - Article

VL - 84

SP - 555

EP - 579

JO - Journal of the Experimental Analysis of Behavior

JF - Journal of the Experimental Analysis of Behavior

SN - 0022-5002

IS - 3

ER -