Executive Functions: Formative Versus Reflective Measurement

Michael Willoughby, Steven J. Holochwost, Zane E. Blanton, Clancy B. Blair

Research output: Contribution to journalArticle

Abstract

The primary objective of this article was to critically evaluate the routine use of confirmatory factor analysis (CFA) for representing an individual's performance across a battery of executive function tasks. A conceptual review and statistical reanalysis of N = 10 studies that used CFA methods of EF tasks was undertaken. Despite evidence of excellent global model fit for CFA models, studies were uniformly characterized by weak correlations among EF tasks and weak to moderate levels of maximal reliability of latent EF constructs. Vanishing tetrad tests raised the possibility that individual tasks may be better conceptualized as causal rather than as effect indicators of the latent construct of EF. The use of CFA methods with EF tasks that are weakly correlated results in latent variables that have poor to modest maximal reliability. This undermines efforts to use latent variables of EF as predictors or outcomes in empirical studies and has implications for theory development. Greater attention to and resolution of the discrepancies between conceptual definitions and statistical approaches that are used to model EF has implications for the long-term value of using EF in research and clinical settings.

Original languageEnglish (US)
Pages (from-to)69-95
Number of pages27
JournalMeasurement: Journal of the International Measurement Confederation
Volume12
Issue number3
DOIs
StatePublished - 2014

Fingerprint

Confirmatory Factor Analysis
Factor analysis
factor analysis
Latent Variables
development theory
Battery
Empirical Study
Discrepancy
Predictors
Model
electric batteries
Evaluate
performance
evidence
predictions

Keywords

  • confirmatory factor analysis
  • executive function
  • formative measurement
  • maximal reliability

ASJC Scopus subject areas

  • Statistics and Probability
  • Applied Mathematics
  • Education

Cite this

Executive Functions : Formative Versus Reflective Measurement. / Willoughby, Michael; Holochwost, Steven J.; Blanton, Zane E.; Blair, Clancy B.

In: Measurement: Journal of the International Measurement Confederation, Vol. 12, No. 3, 2014, p. 69-95.

Research output: Contribution to journalArticle

Willoughby, Michael ; Holochwost, Steven J. ; Blanton, Zane E. ; Blair, Clancy B. / Executive Functions : Formative Versus Reflective Measurement. In: Measurement: Journal of the International Measurement Confederation. 2014 ; Vol. 12, No. 3. pp. 69-95.
@article{e3500166ab004fb093396f7f39f9b16f,
title = "Executive Functions: Formative Versus Reflective Measurement",
abstract = "The primary objective of this article was to critically evaluate the routine use of confirmatory factor analysis (CFA) for representing an individual's performance across a battery of executive function tasks. A conceptual review and statistical reanalysis of N = 10 studies that used CFA methods of EF tasks was undertaken. Despite evidence of excellent global model fit for CFA models, studies were uniformly characterized by weak correlations among EF tasks and weak to moderate levels of maximal reliability of latent EF constructs. Vanishing tetrad tests raised the possibility that individual tasks may be better conceptualized as causal rather than as effect indicators of the latent construct of EF. The use of CFA methods with EF tasks that are weakly correlated results in latent variables that have poor to modest maximal reliability. This undermines efforts to use latent variables of EF as predictors or outcomes in empirical studies and has implications for theory development. Greater attention to and resolution of the discrepancies between conceptual definitions and statistical approaches that are used to model EF has implications for the long-term value of using EF in research and clinical settings.",
keywords = "confirmatory factor analysis, executive function, formative measurement, maximal reliability",
author = "Michael Willoughby and Holochwost, {Steven J.} and Blanton, {Zane E.} and Blair, {Clancy B.}",
year = "2014",
doi = "10.1080/15366367.2014.929453",
language = "English (US)",
volume = "12",
pages = "69--95",
journal = "Measurement: Journal of the International Measurement Confederation",
issn = "0263-2241",
publisher = "Elsevier",
number = "3",

}

TY - JOUR

T1 - Executive Functions

T2 - Formative Versus Reflective Measurement

AU - Willoughby, Michael

AU - Holochwost, Steven J.

AU - Blanton, Zane E.

AU - Blair, Clancy B.

PY - 2014

Y1 - 2014

N2 - The primary objective of this article was to critically evaluate the routine use of confirmatory factor analysis (CFA) for representing an individual's performance across a battery of executive function tasks. A conceptual review and statistical reanalysis of N = 10 studies that used CFA methods of EF tasks was undertaken. Despite evidence of excellent global model fit for CFA models, studies were uniformly characterized by weak correlations among EF tasks and weak to moderate levels of maximal reliability of latent EF constructs. Vanishing tetrad tests raised the possibility that individual tasks may be better conceptualized as causal rather than as effect indicators of the latent construct of EF. The use of CFA methods with EF tasks that are weakly correlated results in latent variables that have poor to modest maximal reliability. This undermines efforts to use latent variables of EF as predictors or outcomes in empirical studies and has implications for theory development. Greater attention to and resolution of the discrepancies between conceptual definitions and statistical approaches that are used to model EF has implications for the long-term value of using EF in research and clinical settings.

AB - The primary objective of this article was to critically evaluate the routine use of confirmatory factor analysis (CFA) for representing an individual's performance across a battery of executive function tasks. A conceptual review and statistical reanalysis of N = 10 studies that used CFA methods of EF tasks was undertaken. Despite evidence of excellent global model fit for CFA models, studies were uniformly characterized by weak correlations among EF tasks and weak to moderate levels of maximal reliability of latent EF constructs. Vanishing tetrad tests raised the possibility that individual tasks may be better conceptualized as causal rather than as effect indicators of the latent construct of EF. The use of CFA methods with EF tasks that are weakly correlated results in latent variables that have poor to modest maximal reliability. This undermines efforts to use latent variables of EF as predictors or outcomes in empirical studies and has implications for theory development. Greater attention to and resolution of the discrepancies between conceptual definitions and statistical approaches that are used to model EF has implications for the long-term value of using EF in research and clinical settings.

KW - confirmatory factor analysis

KW - executive function

KW - formative measurement

KW - maximal reliability

UR - http://www.scopus.com/inward/record.url?scp=84906537323&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84906537323&partnerID=8YFLogxK

U2 - 10.1080/15366367.2014.929453

DO - 10.1080/15366367.2014.929453

M3 - Article

VL - 12

SP - 69

EP - 95

JO - Measurement: Journal of the International Measurement Confederation

JF - Measurement: Journal of the International Measurement Confederation

SN - 0263-2241

IS - 3

ER -