Benchmarks provide common ground for model development

Reply to Logie (2018) and Vandierendonck (2018)

Klaus Oberauer, Stephan Lewandowsky, Edward Awh, Gordon D.A. Brown, Andrew Conway, Nelson Cowan, Christopher Donkin, Simon Farrell, Graham J. Hitch, Mark J. Hurlstone, Wei Ji Ma, Candice C. Morey, Derek Evan Nee, Judith Schweppe, Evie Vergauwe, Geoff Ward

Research output: Contribution to journalLetter

Abstract

We respond to the comments of Logie and Vandierendonck to our article proposing benchmark findings for evaluating theories and models of short-term and working memory. The response focuses on the two main points of criticism: (a) Logie and Vandierendonck argue that the scope of the set of benchmarks is too narrow. We explain why findings on how working memory is used in complex cognition, findings on executive functions, and findings from neuropsychological case studies are currently not included in the benchmarks, and why findings with visual and spatial materials are less prevalent among them. (b) The critics question the usefulness of the benchmarks and their ratings for advancing theory development. We explain why selecting and rating benchmarks is important and justifiable, and acknowledge that the present selection and rating decisions are in need of continuous updating. The usefulness of the benchmarks of all ratings is also enhanced by our concomitant online posting of data for many of these benchmarks.

Original languageEnglish (US)
Pages (from-to)972-977
Number of pages6
JournalPsychological Bulletin
Volume144
Issue number9
DOIs
StatePublished - Sep 1 2018

Fingerprint

Benchmarking
Short-Term Memory
Executive Function
Cognition

Keywords

  • Benchmarks
  • Commentary
  • Executive functions
  • Short-term memory (STM)
  • Working memory

ASJC Scopus subject areas

  • Psychology(all)

Cite this

Oberauer, K., Lewandowsky, S., Awh, E., Brown, G. D. A., Conway, A., Cowan, N., ... Ward, G. (2018). Benchmarks provide common ground for model development: Reply to Logie (2018) and Vandierendonck (2018). Psychological Bulletin, 144(9), 972-977. https://doi.org/10.1037/bul0000165

Benchmarks provide common ground for model development : Reply to Logie (2018) and Vandierendonck (2018). / Oberauer, Klaus; Lewandowsky, Stephan; Awh, Edward; Brown, Gordon D.A.; Conway, Andrew; Cowan, Nelson; Donkin, Christopher; Farrell, Simon; Hitch, Graham J.; Hurlstone, Mark J.; Ma, Wei Ji; Morey, Candice C.; Nee, Derek Evan; Schweppe, Judith; Vergauwe, Evie; Ward, Geoff.

In: Psychological Bulletin, Vol. 144, No. 9, 01.09.2018, p. 972-977.

Research output: Contribution to journalLetter

Oberauer, K, Lewandowsky, S, Awh, E, Brown, GDA, Conway, A, Cowan, N, Donkin, C, Farrell, S, Hitch, GJ, Hurlstone, MJ, Ma, WJ, Morey, CC, Nee, DE, Schweppe, J, Vergauwe, E & Ward, G 2018, 'Benchmarks provide common ground for model development: Reply to Logie (2018) and Vandierendonck (2018)', Psychological Bulletin, vol. 144, no. 9, pp. 972-977. https://doi.org/10.1037/bul0000165
Oberauer, Klaus ; Lewandowsky, Stephan ; Awh, Edward ; Brown, Gordon D.A. ; Conway, Andrew ; Cowan, Nelson ; Donkin, Christopher ; Farrell, Simon ; Hitch, Graham J. ; Hurlstone, Mark J. ; Ma, Wei Ji ; Morey, Candice C. ; Nee, Derek Evan ; Schweppe, Judith ; Vergauwe, Evie ; Ward, Geoff. / Benchmarks provide common ground for model development : Reply to Logie (2018) and Vandierendonck (2018). In: Psychological Bulletin. 2018 ; Vol. 144, No. 9. pp. 972-977.
@article{231f52cd817f454198d8867169a41a10,
title = "Benchmarks provide common ground for model development: Reply to Logie (2018) and Vandierendonck (2018)",
abstract = "We respond to the comments of Logie and Vandierendonck to our article proposing benchmark findings for evaluating theories and models of short-term and working memory. The response focuses on the two main points of criticism: (a) Logie and Vandierendonck argue that the scope of the set of benchmarks is too narrow. We explain why findings on how working memory is used in complex cognition, findings on executive functions, and findings from neuropsychological case studies are currently not included in the benchmarks, and why findings with visual and spatial materials are less prevalent among them. (b) The critics question the usefulness of the benchmarks and their ratings for advancing theory development. We explain why selecting and rating benchmarks is important and justifiable, and acknowledge that the present selection and rating decisions are in need of continuous updating. The usefulness of the benchmarks of all ratings is also enhanced by our concomitant online posting of data for many of these benchmarks.",
keywords = "Benchmarks, Commentary, Executive functions, Short-term memory (STM), Working memory",
author = "Klaus Oberauer and Stephan Lewandowsky and Edward Awh and Brown, {Gordon D.A.} and Andrew Conway and Nelson Cowan and Christopher Donkin and Simon Farrell and Hitch, {Graham J.} and Hurlstone, {Mark J.} and Ma, {Wei Ji} and Morey, {Candice C.} and Nee, {Derek Evan} and Judith Schweppe and Evie Vergauwe and Geoff Ward",
year = "2018",
month = "9",
day = "1",
doi = "10.1037/bul0000165",
language = "English (US)",
volume = "144",
pages = "972--977",
journal = "Psychological Bulletin",
issn = "0033-2909",
publisher = "American Psychological Association Inc.",
number = "9",

}

TY - JOUR

T1 - Benchmarks provide common ground for model development

T2 - Reply to Logie (2018) and Vandierendonck (2018)

AU - Oberauer, Klaus

AU - Lewandowsky, Stephan

AU - Awh, Edward

AU - Brown, Gordon D.A.

AU - Conway, Andrew

AU - Cowan, Nelson

AU - Donkin, Christopher

AU - Farrell, Simon

AU - Hitch, Graham J.

AU - Hurlstone, Mark J.

AU - Ma, Wei Ji

AU - Morey, Candice C.

AU - Nee, Derek Evan

AU - Schweppe, Judith

AU - Vergauwe, Evie

AU - Ward, Geoff

PY - 2018/9/1

Y1 - 2018/9/1

N2 - We respond to the comments of Logie and Vandierendonck to our article proposing benchmark findings for evaluating theories and models of short-term and working memory. The response focuses on the two main points of criticism: (a) Logie and Vandierendonck argue that the scope of the set of benchmarks is too narrow. We explain why findings on how working memory is used in complex cognition, findings on executive functions, and findings from neuropsychological case studies are currently not included in the benchmarks, and why findings with visual and spatial materials are less prevalent among them. (b) The critics question the usefulness of the benchmarks and their ratings for advancing theory development. We explain why selecting and rating benchmarks is important and justifiable, and acknowledge that the present selection and rating decisions are in need of continuous updating. The usefulness of the benchmarks of all ratings is also enhanced by our concomitant online posting of data for many of these benchmarks.

AB - We respond to the comments of Logie and Vandierendonck to our article proposing benchmark findings for evaluating theories and models of short-term and working memory. The response focuses on the two main points of criticism: (a) Logie and Vandierendonck argue that the scope of the set of benchmarks is too narrow. We explain why findings on how working memory is used in complex cognition, findings on executive functions, and findings from neuropsychological case studies are currently not included in the benchmarks, and why findings with visual and spatial materials are less prevalent among them. (b) The critics question the usefulness of the benchmarks and their ratings for advancing theory development. We explain why selecting and rating benchmarks is important and justifiable, and acknowledge that the present selection and rating decisions are in need of continuous updating. The usefulness of the benchmarks of all ratings is also enhanced by our concomitant online posting of data for many of these benchmarks.

KW - Benchmarks

KW - Commentary

KW - Executive functions

KW - Short-term memory (STM)

KW - Working memory

UR - http://www.scopus.com/inward/record.url?scp=85053506388&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85053506388&partnerID=8YFLogxK

U2 - 10.1037/bul0000165

DO - 10.1037/bul0000165

M3 - Letter

VL - 144

SP - 972

EP - 977

JO - Psychological Bulletin

JF - Psychological Bulletin

SN - 0033-2909

IS - 9

ER -