The p-Value Requires Context, Not a Threshold

Research output: Contribution to journalArticle

Abstract

It is widely recognized by statisticians, though not as widely by other researchers, that the p-value cannot be interpreted in isolation, but rather must be considered in the context of certain features of the design and substantive application, such as sample size and meaningful effect size. I consider the setting of the normal mean and highlight the information contained in the p-value in conjunction with the sample size and meaningful effect size. The p-value and sample size jointly yield 95% confidence bounds for the effect of interest, which can be compared to the predetermined meaningful effect size to make inferences about the true effect. I provide simple examples to demonstrate that although the p-value is calculated under the null hypothesis, and thus seemingly may be divorced from the features of the study from which it arises, its interpretation as a measure of evidence requires its contextualization within the study. This implies that any proposal for improved use of the p-value as a measure of the strength of evidence cannot simply be a change to the threshold for significance.

Original languageEnglish (US)
Pages (from-to)115-117
Number of pages3
JournalAmerican Statistician
Volume73
Issue numbersup1
DOIs
StatePublished - Mar 29 2019

Fingerprint

p-Value
Effect Size
Sample Size
Confidence Bounds
Null hypothesis
Isolation
Context
P value
Imply
Demonstrate
Sample size
Effect size
Evidence

Keywords

  • Effect size
  • Sample size
  • Statistical significance

ASJC Scopus subject areas

  • Statistics and Probability
  • Mathematics(all)
  • Statistics, Probability and Uncertainty

Cite this

The p-Value Requires Context, Not a Threshold. / Betensky, Rebecca.

In: American Statistician, Vol. 73, No. sup1, 29.03.2019, p. 115-117.

Research output: Contribution to journalArticle

@article{2e94b5bf85224435b23a3ea24f1b74bb,
title = "The p-Value Requires Context, Not a Threshold",
abstract = "It is widely recognized by statisticians, though not as widely by other researchers, that the p-value cannot be interpreted in isolation, but rather must be considered in the context of certain features of the design and substantive application, such as sample size and meaningful effect size. I consider the setting of the normal mean and highlight the information contained in the p-value in conjunction with the sample size and meaningful effect size. The p-value and sample size jointly yield 95{\%} confidence bounds for the effect of interest, which can be compared to the predetermined meaningful effect size to make inferences about the true effect. I provide simple examples to demonstrate that although the p-value is calculated under the null hypothesis, and thus seemingly may be divorced from the features of the study from which it arises, its interpretation as a measure of evidence requires its contextualization within the study. This implies that any proposal for improved use of the p-value as a measure of the strength of evidence cannot simply be a change to the threshold for significance.",
keywords = "Effect size, Sample size, Statistical significance",
author = "Rebecca Betensky",
year = "2019",
month = "3",
day = "29",
doi = "10.1080/00031305.2018.1529624",
language = "English (US)",
volume = "73",
pages = "115--117",
journal = "American Statistician",
issn = "0003-1305",
publisher = "American Statistical Association",
number = "sup1",

}

TY - JOUR

T1 - The p-Value Requires Context, Not a Threshold

AU - Betensky, Rebecca

PY - 2019/3/29

Y1 - 2019/3/29

N2 - It is widely recognized by statisticians, though not as widely by other researchers, that the p-value cannot be interpreted in isolation, but rather must be considered in the context of certain features of the design and substantive application, such as sample size and meaningful effect size. I consider the setting of the normal mean and highlight the information contained in the p-value in conjunction with the sample size and meaningful effect size. The p-value and sample size jointly yield 95% confidence bounds for the effect of interest, which can be compared to the predetermined meaningful effect size to make inferences about the true effect. I provide simple examples to demonstrate that although the p-value is calculated under the null hypothesis, and thus seemingly may be divorced from the features of the study from which it arises, its interpretation as a measure of evidence requires its contextualization within the study. This implies that any proposal for improved use of the p-value as a measure of the strength of evidence cannot simply be a change to the threshold for significance.

AB - It is widely recognized by statisticians, though not as widely by other researchers, that the p-value cannot be interpreted in isolation, but rather must be considered in the context of certain features of the design and substantive application, such as sample size and meaningful effect size. I consider the setting of the normal mean and highlight the information contained in the p-value in conjunction with the sample size and meaningful effect size. The p-value and sample size jointly yield 95% confidence bounds for the effect of interest, which can be compared to the predetermined meaningful effect size to make inferences about the true effect. I provide simple examples to demonstrate that although the p-value is calculated under the null hypothesis, and thus seemingly may be divorced from the features of the study from which it arises, its interpretation as a measure of evidence requires its contextualization within the study. This implies that any proposal for improved use of the p-value as a measure of the strength of evidence cannot simply be a change to the threshold for significance.

KW - Effect size

KW - Sample size

KW - Statistical significance

UR - http://www.scopus.com/inward/record.url?scp=85063215388&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85063215388&partnerID=8YFLogxK

U2 - 10.1080/00031305.2018.1529624

DO - 10.1080/00031305.2018.1529624

M3 - Article

AN - SCOPUS:85063215388

VL - 73

SP - 115

EP - 117

JO - American Statistician

JF - American Statistician

SN - 0003-1305

IS - sup1

ER -