Researcher requests for inappropriate analysis and reporting

A U.S. Survey of consulting biostatisticians

Min Qi Wang, Alice F. Yan, Ralph Katz

Research output: Contribution to journalArticle

Abstract

Background: Inappropriate analysis and reporting of biomedical research remain a problem despite advances in statistical methods and efforts to educate researchers. Objective: To determine the frequency and severity of requests biostatisticians receive from researchers for inappropriate analysis and reporting of data during statistical consultations. Design: Online survey. Setting: United States. Participants: A randomly drawn sample of 522 American Statistical Association members self-identifying as consulting biostatisticians. Measurements: The Bioethical Issues in Biostatistical Consulting Questionnaire soliciting reports about the frequency and perceived severity of specific requests for inappropriate analysis and reporting. Results: Of 522 consulting biostatisticians contacted, 390 provided sufficient responses: A completion rate of 74.7%. The 4 most frequently reported inappropriate requests rated as "most severe" by at least 20% of the respondents were, in order of frequency, removing or altering some data records to better support the research hypothesis; interpreting the statistical findings on the basis of expectation, not actual results; not reporting the presence of key missing data that might bias the results; and ignoring violations of assumptions that would change results from positive to negative. These requests were reported most often by younger biostatisticians. Limitations: The survey provides information on the reported frequency of inappropriate requests but not on how such requests were handled or whether the requests reflected researchers' maleficence or inadequate knowledge about statistical and research methods. In addition, other inappropriate requests may have been made that were not prespecified in the survey. Conclusion: This survey suggests that researchers frequently make inappropriate requests of their biostatistical consultants regarding the analysis and reporting of their data. Understanding the reasons for these requests and how they are handled requires further study. Primary Funding Source: U.S. Department of Health and Human Services.

Original languageEnglish (US)
Pages (from-to)554-558
Number of pages5
JournalAnnals of internal medicine
Volume169
Issue number8
DOIs
StatePublished - Oct 16 2018

Fingerprint

Research Personnel
Bioethical Issues
Research Design
United States Dept. of Health and Human Services
Consultants
Research
Surveys and Questionnaires
Biomedical Research
Referral and Consultation

ASJC Scopus subject areas

  • Internal Medicine

Cite this

Researcher requests for inappropriate analysis and reporting : A U.S. Survey of consulting biostatisticians. / Wang, Min Qi; Yan, Alice F.; Katz, Ralph.

In: Annals of internal medicine, Vol. 169, No. 8, 16.10.2018, p. 554-558.

Research output: Contribution to journalArticle

@article{5b064a379eb241718c57e3d7e11dfe1b,
title = "Researcher requests for inappropriate analysis and reporting: A U.S. Survey of consulting biostatisticians",
abstract = "Background: Inappropriate analysis and reporting of biomedical research remain a problem despite advances in statistical methods and efforts to educate researchers. Objective: To determine the frequency and severity of requests biostatisticians receive from researchers for inappropriate analysis and reporting of data during statistical consultations. Design: Online survey. Setting: United States. Participants: A randomly drawn sample of 522 American Statistical Association members self-identifying as consulting biostatisticians. Measurements: The Bioethical Issues in Biostatistical Consulting Questionnaire soliciting reports about the frequency and perceived severity of specific requests for inappropriate analysis and reporting. Results: Of 522 consulting biostatisticians contacted, 390 provided sufficient responses: A completion rate of 74.7{\%}. The 4 most frequently reported inappropriate requests rated as {"}most severe{"} by at least 20{\%} of the respondents were, in order of frequency, removing or altering some data records to better support the research hypothesis; interpreting the statistical findings on the basis of expectation, not actual results; not reporting the presence of key missing data that might bias the results; and ignoring violations of assumptions that would change results from positive to negative. These requests were reported most often by younger biostatisticians. Limitations: The survey provides information on the reported frequency of inappropriate requests but not on how such requests were handled or whether the requests reflected researchers' maleficence or inadequate knowledge about statistical and research methods. In addition, other inappropriate requests may have been made that were not prespecified in the survey. Conclusion: This survey suggests that researchers frequently make inappropriate requests of their biostatistical consultants regarding the analysis and reporting of their data. Understanding the reasons for these requests and how they are handled requires further study. Primary Funding Source: U.S. Department of Health and Human Services.",
author = "Wang, {Min Qi} and Yan, {Alice F.} and Ralph Katz",
year = "2018",
month = "10",
day = "16",
doi = "10.7326/M18-1230",
language = "English (US)",
volume = "169",
pages = "554--558",
journal = "Annals of Internal Medicine",
issn = "0003-4819",
publisher = "American College of Physicians",
number = "8",

}

TY - JOUR

T1 - Researcher requests for inappropriate analysis and reporting

T2 - A U.S. Survey of consulting biostatisticians

AU - Wang, Min Qi

AU - Yan, Alice F.

AU - Katz, Ralph

PY - 2018/10/16

Y1 - 2018/10/16

N2 - Background: Inappropriate analysis and reporting of biomedical research remain a problem despite advances in statistical methods and efforts to educate researchers. Objective: To determine the frequency and severity of requests biostatisticians receive from researchers for inappropriate analysis and reporting of data during statistical consultations. Design: Online survey. Setting: United States. Participants: A randomly drawn sample of 522 American Statistical Association members self-identifying as consulting biostatisticians. Measurements: The Bioethical Issues in Biostatistical Consulting Questionnaire soliciting reports about the frequency and perceived severity of specific requests for inappropriate analysis and reporting. Results: Of 522 consulting biostatisticians contacted, 390 provided sufficient responses: A completion rate of 74.7%. The 4 most frequently reported inappropriate requests rated as "most severe" by at least 20% of the respondents were, in order of frequency, removing or altering some data records to better support the research hypothesis; interpreting the statistical findings on the basis of expectation, not actual results; not reporting the presence of key missing data that might bias the results; and ignoring violations of assumptions that would change results from positive to negative. These requests were reported most often by younger biostatisticians. Limitations: The survey provides information on the reported frequency of inappropriate requests but not on how such requests were handled or whether the requests reflected researchers' maleficence or inadequate knowledge about statistical and research methods. In addition, other inappropriate requests may have been made that were not prespecified in the survey. Conclusion: This survey suggests that researchers frequently make inappropriate requests of their biostatistical consultants regarding the analysis and reporting of their data. Understanding the reasons for these requests and how they are handled requires further study. Primary Funding Source: U.S. Department of Health and Human Services.

AB - Background: Inappropriate analysis and reporting of biomedical research remain a problem despite advances in statistical methods and efforts to educate researchers. Objective: To determine the frequency and severity of requests biostatisticians receive from researchers for inappropriate analysis and reporting of data during statistical consultations. Design: Online survey. Setting: United States. Participants: A randomly drawn sample of 522 American Statistical Association members self-identifying as consulting biostatisticians. Measurements: The Bioethical Issues in Biostatistical Consulting Questionnaire soliciting reports about the frequency and perceived severity of specific requests for inappropriate analysis and reporting. Results: Of 522 consulting biostatisticians contacted, 390 provided sufficient responses: A completion rate of 74.7%. The 4 most frequently reported inappropriate requests rated as "most severe" by at least 20% of the respondents were, in order of frequency, removing or altering some data records to better support the research hypothesis; interpreting the statistical findings on the basis of expectation, not actual results; not reporting the presence of key missing data that might bias the results; and ignoring violations of assumptions that would change results from positive to negative. These requests were reported most often by younger biostatisticians. Limitations: The survey provides information on the reported frequency of inappropriate requests but not on how such requests were handled or whether the requests reflected researchers' maleficence or inadequate knowledge about statistical and research methods. In addition, other inappropriate requests may have been made that were not prespecified in the survey. Conclusion: This survey suggests that researchers frequently make inappropriate requests of their biostatistical consultants regarding the analysis and reporting of their data. Understanding the reasons for these requests and how they are handled requires further study. Primary Funding Source: U.S. Department of Health and Human Services.

UR - http://www.scopus.com/inward/record.url?scp=85055071322&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85055071322&partnerID=8YFLogxK

U2 - 10.7326/M18-1230

DO - 10.7326/M18-1230

M3 - Article

VL - 169

SP - 554

EP - 558

JO - Annals of Internal Medicine

JF - Annals of Internal Medicine

SN - 0003-4819

IS - 8

ER -