Maximum differentiation (MAD) competition

A methodology for comparing computational models of perceptual quantities

Zhou Wang, Eero Simoncelli

Research output: Contribution to journalArticle

Abstract

We propose an efficient methodology for comparing computational models of a perceptually discriminable quantity. Rather than comparing model responses to subjective responses on a set of pre-selected stimuli, the stimuli are computer-synthesized so as to optimally distinguish the models. Specifically, given two computational models that take a stimulus as an input and predict a perceptually discriminable quantity, we first synthesize a pair of stimuli that maximize/minimize the response of one model while holding the other fixed. We then repeat this procedure, but with the roles of the two models reversed. Subjective testing on pairs of such synthesized stimuli provides a strong indication of the relative strengths and weaknesses of the two models. Specifically, the model whose extremal stimulus pairs are easier for subjects to discriminate is the better model. Moreover, careful study of the synthesized stimuli may suggest potential ways to improve a model or to combine aspects of multiple models. We demonstrate the methodology for two example perceptual quantities: contrast and image quality.

Original languageEnglish (US)
Article number8
JournalJournal of vision
Volume8
Issue number12
DOIs
StatePublished - Sep 23 2008

Keywords

  • Contrast perception
  • Image quality assessment
  • Maximum differentiation competition
  • Model comparison
  • Perceptual discriminability
  • Stimulus synthesis

ASJC Scopus subject areas

  • Ophthalmology
  • Sensory Systems

Cite this

Maximum differentiation (MAD) competition : A methodology for comparing computational models of perceptual quantities. / Wang, Zhou; Simoncelli, Eero.

In: Journal of vision, Vol. 8, No. 12, 8, 23.09.2008.

Research output: Contribution to journalArticle

@article{ee4264379fc24f6bb8e93bace748c4c9,
title = "Maximum differentiation (MAD) competition: A methodology for comparing computational models of perceptual quantities",
abstract = "We propose an efficient methodology for comparing computational models of a perceptually discriminable quantity. Rather than comparing model responses to subjective responses on a set of pre-selected stimuli, the stimuli are computer-synthesized so as to optimally distinguish the models. Specifically, given two computational models that take a stimulus as an input and predict a perceptually discriminable quantity, we first synthesize a pair of stimuli that maximize/minimize the response of one model while holding the other fixed. We then repeat this procedure, but with the roles of the two models reversed. Subjective testing on pairs of such synthesized stimuli provides a strong indication of the relative strengths and weaknesses of the two models. Specifically, the model whose extremal stimulus pairs are easier for subjects to discriminate is the better model. Moreover, careful study of the synthesized stimuli may suggest potential ways to improve a model or to combine aspects of multiple models. We demonstrate the methodology for two example perceptual quantities: contrast and image quality.",
keywords = "Contrast perception, Image quality assessment, Maximum differentiation competition, Model comparison, Perceptual discriminability, Stimulus synthesis",
author = "Zhou Wang and Eero Simoncelli",
year = "2008",
month = "9",
day = "23",
doi = "10.1167/8.12.8",
language = "English (US)",
volume = "8",
journal = "Journal of vision",
issn = "1534-7362",
number = "12",

}

TY - JOUR

T1 - Maximum differentiation (MAD) competition

T2 - A methodology for comparing computational models of perceptual quantities

AU - Wang, Zhou

AU - Simoncelli, Eero

PY - 2008/9/23

Y1 - 2008/9/23

N2 - We propose an efficient methodology for comparing computational models of a perceptually discriminable quantity. Rather than comparing model responses to subjective responses on a set of pre-selected stimuli, the stimuli are computer-synthesized so as to optimally distinguish the models. Specifically, given two computational models that take a stimulus as an input and predict a perceptually discriminable quantity, we first synthesize a pair of stimuli that maximize/minimize the response of one model while holding the other fixed. We then repeat this procedure, but with the roles of the two models reversed. Subjective testing on pairs of such synthesized stimuli provides a strong indication of the relative strengths and weaknesses of the two models. Specifically, the model whose extremal stimulus pairs are easier for subjects to discriminate is the better model. Moreover, careful study of the synthesized stimuli may suggest potential ways to improve a model or to combine aspects of multiple models. We demonstrate the methodology for two example perceptual quantities: contrast and image quality.

AB - We propose an efficient methodology for comparing computational models of a perceptually discriminable quantity. Rather than comparing model responses to subjective responses on a set of pre-selected stimuli, the stimuli are computer-synthesized so as to optimally distinguish the models. Specifically, given two computational models that take a stimulus as an input and predict a perceptually discriminable quantity, we first synthesize a pair of stimuli that maximize/minimize the response of one model while holding the other fixed. We then repeat this procedure, but with the roles of the two models reversed. Subjective testing on pairs of such synthesized stimuli provides a strong indication of the relative strengths and weaknesses of the two models. Specifically, the model whose extremal stimulus pairs are easier for subjects to discriminate is the better model. Moreover, careful study of the synthesized stimuli may suggest potential ways to improve a model or to combine aspects of multiple models. We demonstrate the methodology for two example perceptual quantities: contrast and image quality.

KW - Contrast perception

KW - Image quality assessment

KW - Maximum differentiation competition

KW - Model comparison

KW - Perceptual discriminability

KW - Stimulus synthesis

UR - http://www.scopus.com/inward/record.url?scp=52649127324&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=52649127324&partnerID=8YFLogxK

U2 - 10.1167/8.12.8

DO - 10.1167/8.12.8

M3 - Article

VL - 8

JO - Journal of vision

JF - Journal of vision

SN - 1534-7362

IS - 12

M1 - 8

ER -