Limitations of proposed signatures of Bayesian confidence

William T. Adler, Wei Ji Ma

Research output: Contribution to journalLetter

Abstract

The Bayesian model of confidence posits that confidence reflects the observer's posterior probability that the decision is correct. Hangya, Sanders, and Kepecs (2016) have proposed that researchers can test the Bayesian model by deriving qualitative signatures of Bayesian confidence (i.e., patterns that one would expect to see if an observer were Bayesian) and looking for those signatures in human or animal data.We examine two proposed signatures, showing that their derivations contain hidden assumptions that limit their applicability and that they are neither necessary nor sufficient conditions for Bayesian confidence. One signature is an average confidence of 0.75 on trials with neutral evidence. This signature holds only when class-conditioned stimulus distributions do not overlap and when internal noise is very low. Another signature is that as stimulus magnitude increases, confidence increases on correct trials but decreases on incorrect trials. This divergence signature holds only when stimulus distributions do not overlap or when noise is high. Navajas et al. (2017) have proposed an alternative form of this signature; we find no indication that this alternative form is expected under Bayesian confidence. Our observations give us pause about the usefulness of the qualitative signatures of Bayesian confidence. To determine the nature of the computations underlying confidence reports, there may be no shortcut to quantitative model comparison.

Original languageEnglish (US)
Pages (from-to)3327-3354
Number of pages28
JournalNeural Computation
Volume30
Issue number12
DOIs
StatePublished - Dec 1 2018

Fingerprint

Noise
Research Personnel
Confidence
Signature
Stimulus

ASJC Scopus subject areas

  • Arts and Humanities (miscellaneous)
  • Cognitive Neuroscience

Cite this

Limitations of proposed signatures of Bayesian confidence. / Adler, William T.; Ma, Wei Ji.

In: Neural Computation, Vol. 30, No. 12, 01.12.2018, p. 3327-3354.

Research output: Contribution to journalLetter

Adler, William T. ; Ma, Wei Ji. / Limitations of proposed signatures of Bayesian confidence. In: Neural Computation. 2018 ; Vol. 30, No. 12. pp. 3327-3354.
@article{1005eecb9ed940ec8866169177b7b68a,
title = "Limitations of proposed signatures of Bayesian confidence",
abstract = "The Bayesian model of confidence posits that confidence reflects the observer's posterior probability that the decision is correct. Hangya, Sanders, and Kepecs (2016) have proposed that researchers can test the Bayesian model by deriving qualitative signatures of Bayesian confidence (i.e., patterns that one would expect to see if an observer were Bayesian) and looking for those signatures in human or animal data.We examine two proposed signatures, showing that their derivations contain hidden assumptions that limit their applicability and that they are neither necessary nor sufficient conditions for Bayesian confidence. One signature is an average confidence of 0.75 on trials with neutral evidence. This signature holds only when class-conditioned stimulus distributions do not overlap and when internal noise is very low. Another signature is that as stimulus magnitude increases, confidence increases on correct trials but decreases on incorrect trials. This divergence signature holds only when stimulus distributions do not overlap or when noise is high. Navajas et al. (2017) have proposed an alternative form of this signature; we find no indication that this alternative form is expected under Bayesian confidence. Our observations give us pause about the usefulness of the qualitative signatures of Bayesian confidence. To determine the nature of the computations underlying confidence reports, there may be no shortcut to quantitative model comparison.",
author = "Adler, {William T.} and Ma, {Wei Ji}",
year = "2018",
month = "12",
day = "1",
doi = "10.1162/neco_a_01141",
language = "English (US)",
volume = "30",
pages = "3327--3354",
journal = "Neural Computation",
issn = "0899-7667",
publisher = "MIT Press Journals",
number = "12",

}

TY - JOUR

T1 - Limitations of proposed signatures of Bayesian confidence

AU - Adler, William T.

AU - Ma, Wei Ji

PY - 2018/12/1

Y1 - 2018/12/1

N2 - The Bayesian model of confidence posits that confidence reflects the observer's posterior probability that the decision is correct. Hangya, Sanders, and Kepecs (2016) have proposed that researchers can test the Bayesian model by deriving qualitative signatures of Bayesian confidence (i.e., patterns that one would expect to see if an observer were Bayesian) and looking for those signatures in human or animal data.We examine two proposed signatures, showing that their derivations contain hidden assumptions that limit their applicability and that they are neither necessary nor sufficient conditions for Bayesian confidence. One signature is an average confidence of 0.75 on trials with neutral evidence. This signature holds only when class-conditioned stimulus distributions do not overlap and when internal noise is very low. Another signature is that as stimulus magnitude increases, confidence increases on correct trials but decreases on incorrect trials. This divergence signature holds only when stimulus distributions do not overlap or when noise is high. Navajas et al. (2017) have proposed an alternative form of this signature; we find no indication that this alternative form is expected under Bayesian confidence. Our observations give us pause about the usefulness of the qualitative signatures of Bayesian confidence. To determine the nature of the computations underlying confidence reports, there may be no shortcut to quantitative model comparison.

AB - The Bayesian model of confidence posits that confidence reflects the observer's posterior probability that the decision is correct. Hangya, Sanders, and Kepecs (2016) have proposed that researchers can test the Bayesian model by deriving qualitative signatures of Bayesian confidence (i.e., patterns that one would expect to see if an observer were Bayesian) and looking for those signatures in human or animal data.We examine two proposed signatures, showing that their derivations contain hidden assumptions that limit their applicability and that they are neither necessary nor sufficient conditions for Bayesian confidence. One signature is an average confidence of 0.75 on trials with neutral evidence. This signature holds only when class-conditioned stimulus distributions do not overlap and when internal noise is very low. Another signature is that as stimulus magnitude increases, confidence increases on correct trials but decreases on incorrect trials. This divergence signature holds only when stimulus distributions do not overlap or when noise is high. Navajas et al. (2017) have proposed an alternative form of this signature; we find no indication that this alternative form is expected under Bayesian confidence. Our observations give us pause about the usefulness of the qualitative signatures of Bayesian confidence. To determine the nature of the computations underlying confidence reports, there may be no shortcut to quantitative model comparison.

UR - http://www.scopus.com/inward/record.url?scp=85057050508&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85057050508&partnerID=8YFLogxK

U2 - 10.1162/neco_a_01141

DO - 10.1162/neco_a_01141

M3 - Letter

VL - 30

SP - 3327

EP - 3354

JO - Neural Computation

JF - Neural Computation

SN - 0899-7667

IS - 12

ER -