Assessing Sensitivity to Unmeasured Confounding Using a Simulated Potential Confounder

Nicole Bohme Carnegie, Masataka Harada, Jennifer L. Hill

Research output: Contribution to journalArticle

Abstract

ABSTRACT: A major obstacle to developing evidenced-based policy is the difficulty of implementing randomized experiments to answer all causal questions of interest. When using a nonexperimental study, it is critical to assess how much the results could be affected by unmeasured confounding. We present a set of graphical and numeric tools to explore the sensitivity of causal estimates to the presence of an unmeasured confounder. We characterize the confounder through two parameters that describe the relationships between (a) the confounder and the treatment assignment and (b) the confounder and the outcome variable. Our approach has two primary advantages over similar approaches that are currently implemented in standard software. First, it can be applied to both continuous and binary treatment variables. Second, our method for binary treatment variables allows the researcher to specify three possible estimands (average treatment effect, effect of the treatment on the treated, effect of the treatment on the controls). These options are all implemented in an R package called treatSens. We demonstrate the efficacy of the method through simulations. We illustrate its potential usefulness in practice in the context of two policy applications.

Original languageEnglish (US)
Pages (from-to)395-420
Number of pages26
JournalJournal of Research on Educational Effectiveness
Volume9
Issue number3
DOIs
StatePublished - Jul 2 2016

Fingerprint

simulation
experiment
software

Keywords

  • causal inference
  • hidden bias
  • omitted variable
  • sensitivity analysis
  • unmeasured confounder

ASJC Scopus subject areas

  • Education

Cite this

Assessing Sensitivity to Unmeasured Confounding Using a Simulated Potential Confounder. / Carnegie, Nicole Bohme; Harada, Masataka; Hill, Jennifer L.

In: Journal of Research on Educational Effectiveness, Vol. 9, No. 3, 02.07.2016, p. 395-420.

Research output: Contribution to journalArticle

@article{406fa856fe34464fb44a621cdea87848,
title = "Assessing Sensitivity to Unmeasured Confounding Using a Simulated Potential Confounder",
abstract = "ABSTRACT: A major obstacle to developing evidenced-based policy is the difficulty of implementing randomized experiments to answer all causal questions of interest. When using a nonexperimental study, it is critical to assess how much the results could be affected by unmeasured confounding. We present a set of graphical and numeric tools to explore the sensitivity of causal estimates to the presence of an unmeasured confounder. We characterize the confounder through two parameters that describe the relationships between (a) the confounder and the treatment assignment and (b) the confounder and the outcome variable. Our approach has two primary advantages over similar approaches that are currently implemented in standard software. First, it can be applied to both continuous and binary treatment variables. Second, our method for binary treatment variables allows the researcher to specify three possible estimands (average treatment effect, effect of the treatment on the treated, effect of the treatment on the controls). These options are all implemented in an R package called treatSens. We demonstrate the efficacy of the method through simulations. We illustrate its potential usefulness in practice in the context of two policy applications.",
keywords = "causal inference, hidden bias, omitted variable, sensitivity analysis, unmeasured confounder",
author = "Carnegie, {Nicole Bohme} and Masataka Harada and Hill, {Jennifer L.}",
year = "2016",
month = "7",
day = "2",
doi = "10.1080/19345747.2015.1078862",
language = "English (US)",
volume = "9",
pages = "395--420",
journal = "Journal of Research on Educational Effectiveness",
issn = "1934-5747",
publisher = "Routledge",
number = "3",

}

TY - JOUR

T1 - Assessing Sensitivity to Unmeasured Confounding Using a Simulated Potential Confounder

AU - Carnegie, Nicole Bohme

AU - Harada, Masataka

AU - Hill, Jennifer L.

PY - 2016/7/2

Y1 - 2016/7/2

N2 - ABSTRACT: A major obstacle to developing evidenced-based policy is the difficulty of implementing randomized experiments to answer all causal questions of interest. When using a nonexperimental study, it is critical to assess how much the results could be affected by unmeasured confounding. We present a set of graphical and numeric tools to explore the sensitivity of causal estimates to the presence of an unmeasured confounder. We characterize the confounder through two parameters that describe the relationships between (a) the confounder and the treatment assignment and (b) the confounder and the outcome variable. Our approach has two primary advantages over similar approaches that are currently implemented in standard software. First, it can be applied to both continuous and binary treatment variables. Second, our method for binary treatment variables allows the researcher to specify three possible estimands (average treatment effect, effect of the treatment on the treated, effect of the treatment on the controls). These options are all implemented in an R package called treatSens. We demonstrate the efficacy of the method through simulations. We illustrate its potential usefulness in practice in the context of two policy applications.

AB - ABSTRACT: A major obstacle to developing evidenced-based policy is the difficulty of implementing randomized experiments to answer all causal questions of interest. When using a nonexperimental study, it is critical to assess how much the results could be affected by unmeasured confounding. We present a set of graphical and numeric tools to explore the sensitivity of causal estimates to the presence of an unmeasured confounder. We characterize the confounder through two parameters that describe the relationships between (a) the confounder and the treatment assignment and (b) the confounder and the outcome variable. Our approach has two primary advantages over similar approaches that are currently implemented in standard software. First, it can be applied to both continuous and binary treatment variables. Second, our method for binary treatment variables allows the researcher to specify three possible estimands (average treatment effect, effect of the treatment on the treated, effect of the treatment on the controls). These options are all implemented in an R package called treatSens. We demonstrate the efficacy of the method through simulations. We illustrate its potential usefulness in practice in the context of two policy applications.

KW - causal inference

KW - hidden bias

KW - omitted variable

KW - sensitivity analysis

KW - unmeasured confounder

UR - http://www.scopus.com/inward/record.url?scp=84958776644&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84958776644&partnerID=8YFLogxK

U2 - 10.1080/19345747.2015.1078862

DO - 10.1080/19345747.2015.1078862

M3 - Article

VL - 9

SP - 395

EP - 420

JO - Journal of Research on Educational Effectiveness

JF - Journal of Research on Educational Effectiveness

SN - 1934-5747

IS - 3

ER -