Testing approaches for overdispersion in poisson regression versus the generalized poisson model

Zhao Yang, James W. Hardin, Cheryl L. Addy, Quang Vuong

    Research output: Contribution to journalArticle

    Abstract

    Overdispersion is a common phenomenon in Poisson modeling, and the negative binomial (NB) model is frequently used to account for overdispersion. Testing approaches (Wald test, likelihood ratio test (LRT), and score test) for overdispersion in the Poisson regression versus the NB model are available. Because the generalized Poisson (GP) model is similar to the NB model, we consider the former as an alternate model for overdispersed count data. The score test has an advantage over the LRT and the Wald test in that the score test only requires that the parameter of interest be estimated under the null hypothesis. This paper proposes a score test for overdispersion based on the GP model and compares the power of the test with the LRT and Wald tests. A simulation study indicates the score test based on asymptotic standard Normal distribution is more appropriate in practical application for higher empirical power, however, it underestimates the nominal significance level, especially in small sample situations, and examples illustrate the results of comparing the candidate tests between the Poisson and GP models. A bootstrap test is also proposed to adjust the underestimation of nominal level in the score statistic when the sample size is small. The simulation study indicates the bootstrap test has significance level closer to nominal size and has uniformly greater power than the score test based on asymptotic standard Normal distribution. From a practical perspective, we suggest that, if the score test gives even a weak indication that the Poisson model is inappropriate, say at the 0.10 significance level, we advise the more accurate bootstrap procedure as a better test for comparing whether the GP model is more appropriate than Poisson model. Finally, the Vuong test is illustrated to choose between GP and NB2 models for the same dataset.

    Original languageEnglish (US)
    Pages (from-to)565-584
    Number of pages20
    JournalBiometrical Journal
    Volume49
    Issue number4
    DOIs
    StatePublished - Aug 2007

    Fingerprint

    Poisson Regression
    Overdispersion
    Score Test
    Poisson Model
    Negative Binomial Model
    Testing
    Wald Test
    Significance level
    Likelihood Ratio Test
    Categorical or nominal
    Standard Normal distribution
    Bootstrap Test
    Siméon Denis Poisson
    Simulation Study
    Score Statistic
    Count Data
    Score test
    Poisson model
    Poisson regression
    Null hypothesis

    Keywords

    • Bootstrap
    • Count data
    • Generalized poisson model
    • Negative binomial model
    • Overdispersion
    • Poisson model
    • Score test
    • Vuong test

    ASJC Scopus subject areas

    • Statistics and Probability

    Cite this

    Testing approaches for overdispersion in poisson regression versus the generalized poisson model. / Yang, Zhao; Hardin, James W.; Addy, Cheryl L.; Vuong, Quang.

    In: Biometrical Journal, Vol. 49, No. 4, 08.2007, p. 565-584.

    Research output: Contribution to journalArticle

    Yang, Zhao ; Hardin, James W. ; Addy, Cheryl L. ; Vuong, Quang. / Testing approaches for overdispersion in poisson regression versus the generalized poisson model. In: Biometrical Journal. 2007 ; Vol. 49, No. 4. pp. 565-584.
    @article{0e33d3a75ba842bdb276ff5e06566c35,
    title = "Testing approaches for overdispersion in poisson regression versus the generalized poisson model",
    abstract = "Overdispersion is a common phenomenon in Poisson modeling, and the negative binomial (NB) model is frequently used to account for overdispersion. Testing approaches (Wald test, likelihood ratio test (LRT), and score test) for overdispersion in the Poisson regression versus the NB model are available. Because the generalized Poisson (GP) model is similar to the NB model, we consider the former as an alternate model for overdispersed count data. The score test has an advantage over the LRT and the Wald test in that the score test only requires that the parameter of interest be estimated under the null hypothesis. This paper proposes a score test for overdispersion based on the GP model and compares the power of the test with the LRT and Wald tests. A simulation study indicates the score test based on asymptotic standard Normal distribution is more appropriate in practical application for higher empirical power, however, it underestimates the nominal significance level, especially in small sample situations, and examples illustrate the results of comparing the candidate tests between the Poisson and GP models. A bootstrap test is also proposed to adjust the underestimation of nominal level in the score statistic when the sample size is small. The simulation study indicates the bootstrap test has significance level closer to nominal size and has uniformly greater power than the score test based on asymptotic standard Normal distribution. From a practical perspective, we suggest that, if the score test gives even a weak indication that the Poisson model is inappropriate, say at the 0.10 significance level, we advise the more accurate bootstrap procedure as a better test for comparing whether the GP model is more appropriate than Poisson model. Finally, the Vuong test is illustrated to choose between GP and NB2 models for the same dataset.",
    keywords = "Bootstrap, Count data, Generalized poisson model, Negative binomial model, Overdispersion, Poisson model, Score test, Vuong test",
    author = "Zhao Yang and Hardin, {James W.} and Addy, {Cheryl L.} and Quang Vuong",
    year = "2007",
    month = "8",
    doi = "10.1002/bimj.200610340",
    language = "English (US)",
    volume = "49",
    pages = "565--584",
    journal = "Biometrical Journal",
    issn = "0323-3847",
    publisher = "Wiley-VCH Verlag",
    number = "4",

    }

    TY - JOUR

    T1 - Testing approaches for overdispersion in poisson regression versus the generalized poisson model

    AU - Yang, Zhao

    AU - Hardin, James W.

    AU - Addy, Cheryl L.

    AU - Vuong, Quang

    PY - 2007/8

    Y1 - 2007/8

    N2 - Overdispersion is a common phenomenon in Poisson modeling, and the negative binomial (NB) model is frequently used to account for overdispersion. Testing approaches (Wald test, likelihood ratio test (LRT), and score test) for overdispersion in the Poisson regression versus the NB model are available. Because the generalized Poisson (GP) model is similar to the NB model, we consider the former as an alternate model for overdispersed count data. The score test has an advantage over the LRT and the Wald test in that the score test only requires that the parameter of interest be estimated under the null hypothesis. This paper proposes a score test for overdispersion based on the GP model and compares the power of the test with the LRT and Wald tests. A simulation study indicates the score test based on asymptotic standard Normal distribution is more appropriate in practical application for higher empirical power, however, it underestimates the nominal significance level, especially in small sample situations, and examples illustrate the results of comparing the candidate tests between the Poisson and GP models. A bootstrap test is also proposed to adjust the underestimation of nominal level in the score statistic when the sample size is small. The simulation study indicates the bootstrap test has significance level closer to nominal size and has uniformly greater power than the score test based on asymptotic standard Normal distribution. From a practical perspective, we suggest that, if the score test gives even a weak indication that the Poisson model is inappropriate, say at the 0.10 significance level, we advise the more accurate bootstrap procedure as a better test for comparing whether the GP model is more appropriate than Poisson model. Finally, the Vuong test is illustrated to choose between GP and NB2 models for the same dataset.

    AB - Overdispersion is a common phenomenon in Poisson modeling, and the negative binomial (NB) model is frequently used to account for overdispersion. Testing approaches (Wald test, likelihood ratio test (LRT), and score test) for overdispersion in the Poisson regression versus the NB model are available. Because the generalized Poisson (GP) model is similar to the NB model, we consider the former as an alternate model for overdispersed count data. The score test has an advantage over the LRT and the Wald test in that the score test only requires that the parameter of interest be estimated under the null hypothesis. This paper proposes a score test for overdispersion based on the GP model and compares the power of the test with the LRT and Wald tests. A simulation study indicates the score test based on asymptotic standard Normal distribution is more appropriate in practical application for higher empirical power, however, it underestimates the nominal significance level, especially in small sample situations, and examples illustrate the results of comparing the candidate tests between the Poisson and GP models. A bootstrap test is also proposed to adjust the underestimation of nominal level in the score statistic when the sample size is small. The simulation study indicates the bootstrap test has significance level closer to nominal size and has uniformly greater power than the score test based on asymptotic standard Normal distribution. From a practical perspective, we suggest that, if the score test gives even a weak indication that the Poisson model is inappropriate, say at the 0.10 significance level, we advise the more accurate bootstrap procedure as a better test for comparing whether the GP model is more appropriate than Poisson model. Finally, the Vuong test is illustrated to choose between GP and NB2 models for the same dataset.

    KW - Bootstrap

    KW - Count data

    KW - Generalized poisson model

    KW - Negative binomial model

    KW - Overdispersion

    KW - Poisson model

    KW - Score test

    KW - Vuong test

    UR - http://www.scopus.com/inward/record.url?scp=34548095721&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=34548095721&partnerID=8YFLogxK

    U2 - 10.1002/bimj.200610340

    DO - 10.1002/bimj.200610340

    M3 - Article

    VL - 49

    SP - 565

    EP - 584

    JO - Biometrical Journal

    JF - Biometrical Journal

    SN - 0323-3847

    IS - 4

    ER -