Comparing prevalence estimates from population-based surveys to inform surveillance using electronic health records

Kathleen S. Tatem, Matthew L. Romo, Katharine H. McVeigh, Pui Ying Chan, Elizabeth Lurie-Moroni, Lorna Thorpe, Sharon E. Perlman

Research output: Contribution to journalArticle

Abstract

Introduction Electronic health record (EHR) systems provide an opportunity to use a novel data source for population health surveillance. Validation studies that compare prevalence estimates from EHRs and surveys most often use difference testing, which can, because of large sample sizes, lead to detection of significant differences that are not meaningful. We explored a novel application of the two one-sided t test (TOST) to assess the equivalence of prevalence estimates in 2 population-based surveys to inform margin selection for validating EHR-based surveillance prevalence estimates derived from large samples. Methods We compared prevalence estimates of health indicators in the 2013 Community Health Survey (CHS) and the 2013-2014 New York City Health and Nutrition Examination Survey (NYC HANES) by using TOST, a 2-tailed t test, and other goodness-offit measures. Results A ±5 percentage-point equivalence margin for a TOST performed well for most health indicators. For health indicators with a prevalence estimate of less than 10% (extreme obesity [CHS, 3.5%; NYC HANES, 5.1%] and serious psychological distress [CHS, 5.2%; NYC HANES, 4.8%]), a ±2.5 percentage-point margin was more consistent with other goodness-of-fit measures than the larger percentage-point margins. Conclusion A TOST with a ±5 percentage-point margin was useful in establishing equivalence, but a ±2.5 percentage-point margin may be appropriate for health indicators with a prevalence estimate of less than 10%. Equivalence testing can guide future efforts to validate EHR data.

Original languageEnglish (US)
JournalPreventing chronic disease
Volume14
Issue number6
DOIs
StatePublished - Jun 1 2017

Fingerprint

Electronic Health Records
Health
Population
Nutrition Surveys
Health Surveys
Population Surveillance
Surveys and Questionnaires
Validation Studies
Information Storage and Retrieval
Sample Size
Obesity
Psychology

ASJC Scopus subject areas

  • Health Policy
  • Public Health, Environmental and Occupational Health

Cite this

Comparing prevalence estimates from population-based surveys to inform surveillance using electronic health records. / Tatem, Kathleen S.; Romo, Matthew L.; McVeigh, Katharine H.; Chan, Pui Ying; Lurie-Moroni, Elizabeth; Thorpe, Lorna; Perlman, Sharon E.

In: Preventing chronic disease, Vol. 14, No. 6, 01.06.2017.

Research output: Contribution to journalArticle

Tatem, Kathleen S. ; Romo, Matthew L. ; McVeigh, Katharine H. ; Chan, Pui Ying ; Lurie-Moroni, Elizabeth ; Thorpe, Lorna ; Perlman, Sharon E. / Comparing prevalence estimates from population-based surveys to inform surveillance using electronic health records. In: Preventing chronic disease. 2017 ; Vol. 14, No. 6.
@article{7362268f88ad4085a2c26d81f5c528ff,
title = "Comparing prevalence estimates from population-based surveys to inform surveillance using electronic health records",
abstract = "Introduction Electronic health record (EHR) systems provide an opportunity to use a novel data source for population health surveillance. Validation studies that compare prevalence estimates from EHRs and surveys most often use difference testing, which can, because of large sample sizes, lead to detection of significant differences that are not meaningful. We explored a novel application of the two one-sided t test (TOST) to assess the equivalence of prevalence estimates in 2 population-based surveys to inform margin selection for validating EHR-based surveillance prevalence estimates derived from large samples. Methods We compared prevalence estimates of health indicators in the 2013 Community Health Survey (CHS) and the 2013-2014 New York City Health and Nutrition Examination Survey (NYC HANES) by using TOST, a 2-tailed t test, and other goodness-offit measures. Results A ±5 percentage-point equivalence margin for a TOST performed well for most health indicators. For health indicators with a prevalence estimate of less than 10{\%} (extreme obesity [CHS, 3.5{\%}; NYC HANES, 5.1{\%}] and serious psychological distress [CHS, 5.2{\%}; NYC HANES, 4.8{\%}]), a ±2.5 percentage-point margin was more consistent with other goodness-of-fit measures than the larger percentage-point margins. Conclusion A TOST with a ±5 percentage-point margin was useful in establishing equivalence, but a ±2.5 percentage-point margin may be appropriate for health indicators with a prevalence estimate of less than 10{\%}. Equivalence testing can guide future efforts to validate EHR data.",
author = "Tatem, {Kathleen S.} and Romo, {Matthew L.} and McVeigh, {Katharine H.} and Chan, {Pui Ying} and Elizabeth Lurie-Moroni and Lorna Thorpe and Perlman, {Sharon E.}",
year = "2017",
month = "6",
day = "1",
doi = "10.5888/pcd14.160516",
language = "English (US)",
volume = "14",
journal = "Preventing chronic disease",
issn = "1545-1151",
publisher = "U.S. Department of Health and Human Services",
number = "6",

}

TY - JOUR

T1 - Comparing prevalence estimates from population-based surveys to inform surveillance using electronic health records

AU - Tatem, Kathleen S.

AU - Romo, Matthew L.

AU - McVeigh, Katharine H.

AU - Chan, Pui Ying

AU - Lurie-Moroni, Elizabeth

AU - Thorpe, Lorna

AU - Perlman, Sharon E.

PY - 2017/6/1

Y1 - 2017/6/1

N2 - Introduction Electronic health record (EHR) systems provide an opportunity to use a novel data source for population health surveillance. Validation studies that compare prevalence estimates from EHRs and surveys most often use difference testing, which can, because of large sample sizes, lead to detection of significant differences that are not meaningful. We explored a novel application of the two one-sided t test (TOST) to assess the equivalence of prevalence estimates in 2 population-based surveys to inform margin selection for validating EHR-based surveillance prevalence estimates derived from large samples. Methods We compared prevalence estimates of health indicators in the 2013 Community Health Survey (CHS) and the 2013-2014 New York City Health and Nutrition Examination Survey (NYC HANES) by using TOST, a 2-tailed t test, and other goodness-offit measures. Results A ±5 percentage-point equivalence margin for a TOST performed well for most health indicators. For health indicators with a prevalence estimate of less than 10% (extreme obesity [CHS, 3.5%; NYC HANES, 5.1%] and serious psychological distress [CHS, 5.2%; NYC HANES, 4.8%]), a ±2.5 percentage-point margin was more consistent with other goodness-of-fit measures than the larger percentage-point margins. Conclusion A TOST with a ±5 percentage-point margin was useful in establishing equivalence, but a ±2.5 percentage-point margin may be appropriate for health indicators with a prevalence estimate of less than 10%. Equivalence testing can guide future efforts to validate EHR data.

AB - Introduction Electronic health record (EHR) systems provide an opportunity to use a novel data source for population health surveillance. Validation studies that compare prevalence estimates from EHRs and surveys most often use difference testing, which can, because of large sample sizes, lead to detection of significant differences that are not meaningful. We explored a novel application of the two one-sided t test (TOST) to assess the equivalence of prevalence estimates in 2 population-based surveys to inform margin selection for validating EHR-based surveillance prevalence estimates derived from large samples. Methods We compared prevalence estimates of health indicators in the 2013 Community Health Survey (CHS) and the 2013-2014 New York City Health and Nutrition Examination Survey (NYC HANES) by using TOST, a 2-tailed t test, and other goodness-offit measures. Results A ±5 percentage-point equivalence margin for a TOST performed well for most health indicators. For health indicators with a prevalence estimate of less than 10% (extreme obesity [CHS, 3.5%; NYC HANES, 5.1%] and serious psychological distress [CHS, 5.2%; NYC HANES, 4.8%]), a ±2.5 percentage-point margin was more consistent with other goodness-of-fit measures than the larger percentage-point margins. Conclusion A TOST with a ±5 percentage-point margin was useful in establishing equivalence, but a ±2.5 percentage-point margin may be appropriate for health indicators with a prevalence estimate of less than 10%. Equivalence testing can guide future efforts to validate EHR data.

UR - http://www.scopus.com/inward/record.url?scp=85020481905&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85020481905&partnerID=8YFLogxK

U2 - 10.5888/pcd14.160516

DO - 10.5888/pcd14.160516

M3 - Article

VL - 14

JO - Preventing chronic disease

JF - Preventing chronic disease

SN - 1545-1151

IS - 6

ER -