A detailed comparison of optimality and simplicity in perceptual decision making

Shan Shen, Wei Ji Ma

Research output: Contribution to journalArticle

Abstract

Two prominent ideas in the study of decision making have been that organisms behave near-optimally, and that they use simple heuristic rules. These principles might be operating in different types of tasks, but this possibility cannot be fully investigated without a direct, rigorous comparison within a single task. Such a comparison was lacking in most previous studies, because (a) the optimal decision rule was simple, (b) no simple suboptimal rules were considered, (c) it was unclear what was optimal, or (d) a simple rule could closely approximate the optimal rule. Here, we used a perceptual decision-making task in which the optimal decision rule is well-defined and complex, and makes qualitatively distinct predictions from many simple suboptimal rules. We find that all simple rules tested fail to describe human behavior, that the optimal rule accounts well for the data, and that several complex suboptimal rules are indistinguishable from the optimal one. Moreover, we found evidence that the optimal model is close to the true model: First, the better the trial-to-trial predictions of a suboptimal model agree with those of the optimal model, the better that suboptimal model fits; second, our estimate of the Kullback- Leibler divergence between the optimal model and the true model is not significantly different from zero. When observers receive no feedback, the optimal model still describes behavior best, suggesting that sensory uncertainty is implicitly represented and taken into account. Beyond the task and models studied here, our results have implications for best practices of model comparison.

Original languageEnglish (US)
Pages (from-to)152-180
Number of pages29
JournalPsychological Review
Volume123
Issue number4
DOIs
StatePublished - Jul 1 2016

Fingerprint

Decision Making
Practice Guidelines
Uncertainty
Heuristics

Keywords

  • Ideal observer
  • Model comparison
  • Optimality
  • Perception
  • Visual search

ASJC Scopus subject areas

  • Psychology(all)

Cite this

A detailed comparison of optimality and simplicity in perceptual decision making. / Shen, Shan; Ma, Wei Ji.

In: Psychological Review, Vol. 123, No. 4, 01.07.2016, p. 152-180.

Research output: Contribution to journalArticle

@article{4f39255535964757a3da765953fa1c9c,
title = "A detailed comparison of optimality and simplicity in perceptual decision making",
abstract = "Two prominent ideas in the study of decision making have been that organisms behave near-optimally, and that they use simple heuristic rules. These principles might be operating in different types of tasks, but this possibility cannot be fully investigated without a direct, rigorous comparison within a single task. Such a comparison was lacking in most previous studies, because (a) the optimal decision rule was simple, (b) no simple suboptimal rules were considered, (c) it was unclear what was optimal, or (d) a simple rule could closely approximate the optimal rule. Here, we used a perceptual decision-making task in which the optimal decision rule is well-defined and complex, and makes qualitatively distinct predictions from many simple suboptimal rules. We find that all simple rules tested fail to describe human behavior, that the optimal rule accounts well for the data, and that several complex suboptimal rules are indistinguishable from the optimal one. Moreover, we found evidence that the optimal model is close to the true model: First, the better the trial-to-trial predictions of a suboptimal model agree with those of the optimal model, the better that suboptimal model fits; second, our estimate of the Kullback- Leibler divergence between the optimal model and the true model is not significantly different from zero. When observers receive no feedback, the optimal model still describes behavior best, suggesting that sensory uncertainty is implicitly represented and taken into account. Beyond the task and models studied here, our results have implications for best practices of model comparison.",
keywords = "Ideal observer, Model comparison, Optimality, Perception, Visual search",
author = "Shan Shen and Ma, {Wei Ji}",
year = "2016",
month = "7",
day = "1",
doi = "10.1037/rev0000028",
language = "English (US)",
volume = "123",
pages = "152--180",
journal = "Psychological Review",
issn = "0033-295X",
publisher = "American Psychological Association Inc.",
number = "4",

}

TY - JOUR

T1 - A detailed comparison of optimality and simplicity in perceptual decision making

AU - Shen, Shan

AU - Ma, Wei Ji

PY - 2016/7/1

Y1 - 2016/7/1

N2 - Two prominent ideas in the study of decision making have been that organisms behave near-optimally, and that they use simple heuristic rules. These principles might be operating in different types of tasks, but this possibility cannot be fully investigated without a direct, rigorous comparison within a single task. Such a comparison was lacking in most previous studies, because (a) the optimal decision rule was simple, (b) no simple suboptimal rules were considered, (c) it was unclear what was optimal, or (d) a simple rule could closely approximate the optimal rule. Here, we used a perceptual decision-making task in which the optimal decision rule is well-defined and complex, and makes qualitatively distinct predictions from many simple suboptimal rules. We find that all simple rules tested fail to describe human behavior, that the optimal rule accounts well for the data, and that several complex suboptimal rules are indistinguishable from the optimal one. Moreover, we found evidence that the optimal model is close to the true model: First, the better the trial-to-trial predictions of a suboptimal model agree with those of the optimal model, the better that suboptimal model fits; second, our estimate of the Kullback- Leibler divergence between the optimal model and the true model is not significantly different from zero. When observers receive no feedback, the optimal model still describes behavior best, suggesting that sensory uncertainty is implicitly represented and taken into account. Beyond the task and models studied here, our results have implications for best practices of model comparison.

AB - Two prominent ideas in the study of decision making have been that organisms behave near-optimally, and that they use simple heuristic rules. These principles might be operating in different types of tasks, but this possibility cannot be fully investigated without a direct, rigorous comparison within a single task. Such a comparison was lacking in most previous studies, because (a) the optimal decision rule was simple, (b) no simple suboptimal rules were considered, (c) it was unclear what was optimal, or (d) a simple rule could closely approximate the optimal rule. Here, we used a perceptual decision-making task in which the optimal decision rule is well-defined and complex, and makes qualitatively distinct predictions from many simple suboptimal rules. We find that all simple rules tested fail to describe human behavior, that the optimal rule accounts well for the data, and that several complex suboptimal rules are indistinguishable from the optimal one. Moreover, we found evidence that the optimal model is close to the true model: First, the better the trial-to-trial predictions of a suboptimal model agree with those of the optimal model, the better that suboptimal model fits; second, our estimate of the Kullback- Leibler divergence between the optimal model and the true model is not significantly different from zero. When observers receive no feedback, the optimal model still describes behavior best, suggesting that sensory uncertainty is implicitly represented and taken into account. Beyond the task and models studied here, our results have implications for best practices of model comparison.

KW - Ideal observer

KW - Model comparison

KW - Optimality

KW - Perception

KW - Visual search

UR - http://www.scopus.com/inward/record.url?scp=84967263107&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84967263107&partnerID=8YFLogxK

U2 - 10.1037/rev0000028

DO - 10.1037/rev0000028

M3 - Article

C2 - 27177259

AN - SCOPUS:84967263107

VL - 123

SP - 152

EP - 180

JO - Psychological Review

JF - Psychological Review

SN - 0033-295X

IS - 4

ER -