An examination of methods for sample size recalculation during an experiment

Rebecca Betensky, Camlin Tierney

Research output: Contribution to journalArticle

Abstract

In designing experiments, investigators frequently can specify an important effect that they wish to detect with high power, without the ability to provide an equally certain assessment of the variance of the response. If the experiment is designed based on a guess of the variance, an under-powered study may result. To remedy this problem, there have been several procedures proposed that obtain estimates of the variance from the data as they accrue and then recalculate the sample size accordingly. One class of procedures is fully sequential in that it assesses after each response whether the current sample size yields the desired power based on the current estimate of the variance. This approach is efficient, but it is not practical or advisable in many situations. Another class of procedures involves only two or three stages of sampling and recalculates the sample size based on the observed variance at designated times, perhaps coinciding with interim efficacy analyses. The two-stage approach can result in substantial oversampling, but it is feasible in many situations, whereas the three-stage approach corrects the problem of oversampling, but is less feasible. We propose a procedure that aims to combine the advantages of both the fully sequential and the two-stage approaches. This quasi-sequential procedure involves only two stages of sampling and it applies the stopping rule from the fully sequential procedure to data beyond the initial sample which we obtain via multiple imputation. We show through simulations that when the initial sample size is substantially less than the correct sample size, the mean squared error of the final sample size calculated from the quasi-sequential procedure can be considerably less than that from the two-stage procedure. We compare the distributions of these recalculated sample sizes and discuss our findings for alternative procedures, as well.

Original languageEnglish (US)
Pages (from-to)2587-2598
Number of pages12
JournalStatistics in Medicine
Volume16
Issue number22
DOIs
StatePublished - Nov 30 1997

Fingerprint

Sample Size
Sequential Procedure
Experiment
Oversampling
Two-stage Procedure
Multiple Imputation
Stopping Rule
Guess
Mean Squared Error
Estimate
High Power
Efficacy
Research Personnel
Alternatives
Simulation

ASJC Scopus subject areas

  • Epidemiology
  • Statistics and Probability

Cite this

An examination of methods for sample size recalculation during an experiment. / Betensky, Rebecca; Tierney, Camlin.

In: Statistics in Medicine, Vol. 16, No. 22, 30.11.1997, p. 2587-2598.

Research output: Contribution to journalArticle

@article{b6a60ce701654674a169aee1ec9c0561,
title = "An examination of methods for sample size recalculation during an experiment",
abstract = "In designing experiments, investigators frequently can specify an important effect that they wish to detect with high power, without the ability to provide an equally certain assessment of the variance of the response. If the experiment is designed based on a guess of the variance, an under-powered study may result. To remedy this problem, there have been several procedures proposed that obtain estimates of the variance from the data as they accrue and then recalculate the sample size accordingly. One class of procedures is fully sequential in that it assesses after each response whether the current sample size yields the desired power based on the current estimate of the variance. This approach is efficient, but it is not practical or advisable in many situations. Another class of procedures involves only two or three stages of sampling and recalculates the sample size based on the observed variance at designated times, perhaps coinciding with interim efficacy analyses. The two-stage approach can result in substantial oversampling, but it is feasible in many situations, whereas the three-stage approach corrects the problem of oversampling, but is less feasible. We propose a procedure that aims to combine the advantages of both the fully sequential and the two-stage approaches. This quasi-sequential procedure involves only two stages of sampling and it applies the stopping rule from the fully sequential procedure to data beyond the initial sample which we obtain via multiple imputation. We show through simulations that when the initial sample size is substantially less than the correct sample size, the mean squared error of the final sample size calculated from the quasi-sequential procedure can be considerably less than that from the two-stage procedure. We compare the distributions of these recalculated sample sizes and discuss our findings for alternative procedures, as well.",
author = "Rebecca Betensky and Camlin Tierney",
year = "1997",
month = "11",
day = "30",
doi = "10.1002/(SICI)1097-0258(19971130)16:22<2587::AID-SIM687>3.0.CO;2-5",
language = "English (US)",
volume = "16",
pages = "2587--2598",
journal = "Statistics in Medicine",
issn = "0277-6715",
publisher = "John Wiley and Sons Ltd",
number = "22",

}

TY - JOUR

T1 - An examination of methods for sample size recalculation during an experiment

AU - Betensky, Rebecca

AU - Tierney, Camlin

PY - 1997/11/30

Y1 - 1997/11/30

N2 - In designing experiments, investigators frequently can specify an important effect that they wish to detect with high power, without the ability to provide an equally certain assessment of the variance of the response. If the experiment is designed based on a guess of the variance, an under-powered study may result. To remedy this problem, there have been several procedures proposed that obtain estimates of the variance from the data as they accrue and then recalculate the sample size accordingly. One class of procedures is fully sequential in that it assesses after each response whether the current sample size yields the desired power based on the current estimate of the variance. This approach is efficient, but it is not practical or advisable in many situations. Another class of procedures involves only two or three stages of sampling and recalculates the sample size based on the observed variance at designated times, perhaps coinciding with interim efficacy analyses. The two-stage approach can result in substantial oversampling, but it is feasible in many situations, whereas the three-stage approach corrects the problem of oversampling, but is less feasible. We propose a procedure that aims to combine the advantages of both the fully sequential and the two-stage approaches. This quasi-sequential procedure involves only two stages of sampling and it applies the stopping rule from the fully sequential procedure to data beyond the initial sample which we obtain via multiple imputation. We show through simulations that when the initial sample size is substantially less than the correct sample size, the mean squared error of the final sample size calculated from the quasi-sequential procedure can be considerably less than that from the two-stage procedure. We compare the distributions of these recalculated sample sizes and discuss our findings for alternative procedures, as well.

AB - In designing experiments, investigators frequently can specify an important effect that they wish to detect with high power, without the ability to provide an equally certain assessment of the variance of the response. If the experiment is designed based on a guess of the variance, an under-powered study may result. To remedy this problem, there have been several procedures proposed that obtain estimates of the variance from the data as they accrue and then recalculate the sample size accordingly. One class of procedures is fully sequential in that it assesses after each response whether the current sample size yields the desired power based on the current estimate of the variance. This approach is efficient, but it is not practical or advisable in many situations. Another class of procedures involves only two or three stages of sampling and recalculates the sample size based on the observed variance at designated times, perhaps coinciding with interim efficacy analyses. The two-stage approach can result in substantial oversampling, but it is feasible in many situations, whereas the three-stage approach corrects the problem of oversampling, but is less feasible. We propose a procedure that aims to combine the advantages of both the fully sequential and the two-stage approaches. This quasi-sequential procedure involves only two stages of sampling and it applies the stopping rule from the fully sequential procedure to data beyond the initial sample which we obtain via multiple imputation. We show through simulations that when the initial sample size is substantially less than the correct sample size, the mean squared error of the final sample size calculated from the quasi-sequential procedure can be considerably less than that from the two-stage procedure. We compare the distributions of these recalculated sample sizes and discuss our findings for alternative procedures, as well.

UR - http://www.scopus.com/inward/record.url?scp=0030696126&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0030696126&partnerID=8YFLogxK

U2 - 10.1002/(SICI)1097-0258(19971130)16:22<2587::AID-SIM687>3.0.CO;2-5

DO - 10.1002/(SICI)1097-0258(19971130)16:22<2587::AID-SIM687>3.0.CO;2-5

M3 - Article

C2 - 9403958

AN - SCOPUS:0030696126

VL - 16

SP - 2587

EP - 2598

JO - Statistics in Medicine

JF - Statistics in Medicine

SN - 0277-6715

IS - 22

ER -