Does Regression Produce Representative Estimates of Causal Effects?

Peter M. Aronow, Cyrus Samii

    Research output: Contribution to journalArticle

    Abstract

    With an unrepresentative sample, the estimate of a causal effect may fail to characterize how effects operate in the population of interest. What is less well understood is that conventional estimation practices for observational studies may produce the same problem even with a representative sample. Causal effects estimated via multiple regression differentially weight each unit's contribution. The "effective sample" that regression uses to generate the estimate may bear little resemblance to the population of interest, and the results may be nonrepresentative in a manner similar to what quasi-experimental methods or experiments with convenience samples produce. There is no general external validity basis for preferring multiple regression on representative samples over quasi-experimental or experimental methods. We show how to estimate the "multiple regression weights" that allow one to study the effective sample. We discuss alternative approaches that, under certain conditions, recover representative average causal effects. The requisite conditions cannot always be met.

    Original languageEnglish (US)
    Pages (from-to)250-267
    Number of pages18
    JournalAmerican Journal of Political Science
    Volume60
    Issue number1
    DOIs
    StatePublished - Jan 1 2016

    Fingerprint

    regression
    experiment

    ASJC Scopus subject areas

    • Sociology and Political Science

    Cite this

    Does Regression Produce Representative Estimates of Causal Effects? / Aronow, Peter M.; Samii, Cyrus.

    In: American Journal of Political Science, Vol. 60, No. 1, 01.01.2016, p. 250-267.

    Research output: Contribution to journalArticle

    Aronow, Peter M. ; Samii, Cyrus. / Does Regression Produce Representative Estimates of Causal Effects?. In: American Journal of Political Science. 2016 ; Vol. 60, No. 1. pp. 250-267.
    @article{001132f434284726bc9773f948a49e74,
    title = "Does Regression Produce Representative Estimates of Causal Effects?",
    abstract = "With an unrepresentative sample, the estimate of a causal effect may fail to characterize how effects operate in the population of interest. What is less well understood is that conventional estimation practices for observational studies may produce the same problem even with a representative sample. Causal effects estimated via multiple regression differentially weight each unit's contribution. The {"}effective sample{"} that regression uses to generate the estimate may bear little resemblance to the population of interest, and the results may be nonrepresentative in a manner similar to what quasi-experimental methods or experiments with convenience samples produce. There is no general external validity basis for preferring multiple regression on representative samples over quasi-experimental or experimental methods. We show how to estimate the {"}multiple regression weights{"} that allow one to study the effective sample. We discuss alternative approaches that, under certain conditions, recover representative average causal effects. The requisite conditions cannot always be met.",
    author = "Aronow, {Peter M.} and Cyrus Samii",
    year = "2016",
    month = "1",
    day = "1",
    doi = "10.1111/ajps.12185",
    language = "English (US)",
    volume = "60",
    pages = "250--267",
    journal = "American Journal of Political Science",
    issn = "0092-5853",
    publisher = "Wiley-Blackwell",
    number = "1",

    }

    TY - JOUR

    T1 - Does Regression Produce Representative Estimates of Causal Effects?

    AU - Aronow, Peter M.

    AU - Samii, Cyrus

    PY - 2016/1/1

    Y1 - 2016/1/1

    N2 - With an unrepresentative sample, the estimate of a causal effect may fail to characterize how effects operate in the population of interest. What is less well understood is that conventional estimation practices for observational studies may produce the same problem even with a representative sample. Causal effects estimated via multiple regression differentially weight each unit's contribution. The "effective sample" that regression uses to generate the estimate may bear little resemblance to the population of interest, and the results may be nonrepresentative in a manner similar to what quasi-experimental methods or experiments with convenience samples produce. There is no general external validity basis for preferring multiple regression on representative samples over quasi-experimental or experimental methods. We show how to estimate the "multiple regression weights" that allow one to study the effective sample. We discuss alternative approaches that, under certain conditions, recover representative average causal effects. The requisite conditions cannot always be met.

    AB - With an unrepresentative sample, the estimate of a causal effect may fail to characterize how effects operate in the population of interest. What is less well understood is that conventional estimation practices for observational studies may produce the same problem even with a representative sample. Causal effects estimated via multiple regression differentially weight each unit's contribution. The "effective sample" that regression uses to generate the estimate may bear little resemblance to the population of interest, and the results may be nonrepresentative in a manner similar to what quasi-experimental methods or experiments with convenience samples produce. There is no general external validity basis for preferring multiple regression on representative samples over quasi-experimental or experimental methods. We show how to estimate the "multiple regression weights" that allow one to study the effective sample. We discuss alternative approaches that, under certain conditions, recover representative average causal effects. The requisite conditions cannot always be met.

    UR - http://www.scopus.com/inward/record.url?scp=84953838713&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=84953838713&partnerID=8YFLogxK

    U2 - 10.1111/ajps.12185

    DO - 10.1111/ajps.12185

    M3 - Article

    AN - SCOPUS:84953838713

    VL - 60

    SP - 250

    EP - 267

    JO - American Journal of Political Science

    JF - American Journal of Political Science

    SN - 0092-5853

    IS - 1

    ER -