Biomarker validation with an imperfect reference: Issues and bounds

Sarah C. Emerson, Sushrut S. Waikar, Claudio Fuentes, Joseph V. Bonventre, Rebecca Betensky

Research output: Contribution to journalArticle

Abstract

Motivated by the goal of evaluating a biomarker for acute kidney injury, we consider the problem of assessing operating characteristics for a new biomarker when a true gold standard for disease status is unavailable. In this case, the biomarker is typically compared to another imperfect reference test, and this comparison is used to estimate the performance of the new biomarker. However, errors made by the reference test can bias assessment of the new test. Analysis methods like latent class analysis have been proposed to address this issue, generally employing some strong and unverifiable assumptions regarding the relationship between the new biomarker and the reference test. We investigate the conditional independence assumption that is present in many such approaches and show that for a given set of observed data, conditional independence is only possible for a restricted range of disease prevalence values. We explore the information content of the comparison between the new biomarker and the reference test, and give bounds for the true sensitivity and specificity of the new test when operating characteristics for the reference test are known. We demonstrate that in some cases these bounds may be tight enough to provide useful information, but in other cases these bounds may be quite wide.

Original languageEnglish (US)
Pages (from-to)2933-2945
Number of pages13
JournalStatistical Methods in Medical Research
Volume27
Issue number10
DOIs
StatePublished - Oct 1 2018

Fingerprint

Biomarkers
Imperfect
Conditional Independence
Operating Characteristics
Latent Class Analysis
Acute Kidney Injury
Information Content
Kidney
Gold
Acute
Specificity
Sensitivity and Specificity
Estimate
Range of data
Demonstrate

Keywords

  • Biomarkers
  • conditional independence
  • diagnostic tests
  • imperfect reference
  • sensitivity
  • specificity

ASJC Scopus subject areas

  • Epidemiology
  • Statistics and Probability
  • Health Information Management

Cite this

Biomarker validation with an imperfect reference : Issues and bounds. / Emerson, Sarah C.; Waikar, Sushrut S.; Fuentes, Claudio; Bonventre, Joseph V.; Betensky, Rebecca.

In: Statistical Methods in Medical Research, Vol. 27, No. 10, 01.10.2018, p. 2933-2945.

Research output: Contribution to journalArticle

Emerson, Sarah C. ; Waikar, Sushrut S. ; Fuentes, Claudio ; Bonventre, Joseph V. ; Betensky, Rebecca. / Biomarker validation with an imperfect reference : Issues and bounds. In: Statistical Methods in Medical Research. 2018 ; Vol. 27, No. 10. pp. 2933-2945.
@article{e78d103c2d0a43ce98fa596c74f47c57,
title = "Biomarker validation with an imperfect reference: Issues and bounds",
abstract = "Motivated by the goal of evaluating a biomarker for acute kidney injury, we consider the problem of assessing operating characteristics for a new biomarker when a true gold standard for disease status is unavailable. In this case, the biomarker is typically compared to another imperfect reference test, and this comparison is used to estimate the performance of the new biomarker. However, errors made by the reference test can bias assessment of the new test. Analysis methods like latent class analysis have been proposed to address this issue, generally employing some strong and unverifiable assumptions regarding the relationship between the new biomarker and the reference test. We investigate the conditional independence assumption that is present in many such approaches and show that for a given set of observed data, conditional independence is only possible for a restricted range of disease prevalence values. We explore the information content of the comparison between the new biomarker and the reference test, and give bounds for the true sensitivity and specificity of the new test when operating characteristics for the reference test are known. We demonstrate that in some cases these bounds may be tight enough to provide useful information, but in other cases these bounds may be quite wide.",
keywords = "Biomarkers, conditional independence, diagnostic tests, imperfect reference, sensitivity, specificity",
author = "Emerson, {Sarah C.} and Waikar, {Sushrut S.} and Claudio Fuentes and Bonventre, {Joseph V.} and Rebecca Betensky",
year = "2018",
month = "10",
day = "1",
doi = "10.1177/0962280216689806",
language = "English (US)",
volume = "27",
pages = "2933--2945",
journal = "Statistical Methods in Medical Research",
issn = "0962-2802",
publisher = "SAGE Publications Ltd",
number = "10",

}

TY - JOUR

T1 - Biomarker validation with an imperfect reference

T2 - Issues and bounds

AU - Emerson, Sarah C.

AU - Waikar, Sushrut S.

AU - Fuentes, Claudio

AU - Bonventre, Joseph V.

AU - Betensky, Rebecca

PY - 2018/10/1

Y1 - 2018/10/1

N2 - Motivated by the goal of evaluating a biomarker for acute kidney injury, we consider the problem of assessing operating characteristics for a new biomarker when a true gold standard for disease status is unavailable. In this case, the biomarker is typically compared to another imperfect reference test, and this comparison is used to estimate the performance of the new biomarker. However, errors made by the reference test can bias assessment of the new test. Analysis methods like latent class analysis have been proposed to address this issue, generally employing some strong and unverifiable assumptions regarding the relationship between the new biomarker and the reference test. We investigate the conditional independence assumption that is present in many such approaches and show that for a given set of observed data, conditional independence is only possible for a restricted range of disease prevalence values. We explore the information content of the comparison between the new biomarker and the reference test, and give bounds for the true sensitivity and specificity of the new test when operating characteristics for the reference test are known. We demonstrate that in some cases these bounds may be tight enough to provide useful information, but in other cases these bounds may be quite wide.

AB - Motivated by the goal of evaluating a biomarker for acute kidney injury, we consider the problem of assessing operating characteristics for a new biomarker when a true gold standard for disease status is unavailable. In this case, the biomarker is typically compared to another imperfect reference test, and this comparison is used to estimate the performance of the new biomarker. However, errors made by the reference test can bias assessment of the new test. Analysis methods like latent class analysis have been proposed to address this issue, generally employing some strong and unverifiable assumptions regarding the relationship between the new biomarker and the reference test. We investigate the conditional independence assumption that is present in many such approaches and show that for a given set of observed data, conditional independence is only possible for a restricted range of disease prevalence values. We explore the information content of the comparison between the new biomarker and the reference test, and give bounds for the true sensitivity and specificity of the new test when operating characteristics for the reference test are known. We demonstrate that in some cases these bounds may be tight enough to provide useful information, but in other cases these bounds may be quite wide.

KW - Biomarkers

KW - conditional independence

KW - diagnostic tests

KW - imperfect reference

KW - sensitivity

KW - specificity

UR - http://www.scopus.com/inward/record.url?scp=85043471004&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85043471004&partnerID=8YFLogxK

U2 - 10.1177/0962280216689806

DO - 10.1177/0962280216689806

M3 - Article

C2 - 28166709

AN - SCOPUS:85043471004

VL - 27

SP - 2933

EP - 2945

JO - Statistical Methods in Medical Research

JF - Statistical Methods in Medical Research

SN - 0962-2802

IS - 10

ER -