Interobserver agreement in dysplasia grading

Toward an enhanced gold standard for clinical pathology trials

Paul M. Speight, Timothy J. Abram, Pierre N. Floriano, Robert James, Julie Vick, Martin H. Thornhill, Craig Murdoch, Christine Freeman, Anne M. Hegarty, Katy D'Apice, A. Ross Kerr, Joan Phelan, Patricia Corby, Ismael Khouly, Nadarajah Vigneswaran, Jerry Bouquot, Nagi M. Demian, Y. Etan Weinstock, Spencer W. Redding, Stephanie Rowan & 4 others Chih Ko Yeh, H. Stan McGuff, Frank R. Miller, John McDevitt

Research output: Contribution to journalArticle

Abstract

Objective. Interobserver agreement in the context of oral epithelial dysplasia (OED) grading has been notoriously unreliable and can impose barriers for developing new molecular markers and diagnostic technologies. This paper aimed to report the details of a 3-stage histopathology review and adjudication process with the goal of achieving a consensus histopathologic diagnosis of each biopsy. Study Design. Two adjacent serial histologic sections of oral lesions from 846 patients were independently scored by 2 different pathologists from a pool of 4. In instances where the original 2 pathologists disagreed, a third, independent adjudicating pathologist conducted a review of both sections. If a majority agreement was not achieved, the third stage involved a face-to-face consensus review. Results. Individual pathologist pair κ values ranged from 0.251 to 0.706 (fair-good) before the 3-stage review process. During the initial review phase, the 2 pathologists agreed on a diagnosis for 69.9% of the cases. After the adjudication review by a third pathologist, an additional 22.8% of cases were given a consensus diagnosis (agreement of 2 out of 3 pathologists). After the face-to-face review, the remaining 7.3% of cases had a consensus diagnosis. Conclusions. The use of the defined protocol resulted in a substantial increase (30%) in diagnostic agreement and has the potential to improve the level of agreement for establishing gold standards for studies based on histopathologic diagnosis.

Original languageEnglish (US)
Pages (from-to)474-482.e2
JournalOral Surgery, Oral Medicine, Oral Pathology and Oral Radiology
Volume120
Issue number4
DOIs
StatePublished - Oct 1 2015

Fingerprint

Clinical Pathology
Clinical Trials
Molecular Pathology
Pathologists
Technology
Biopsy

ASJC Scopus subject areas

  • Surgery
  • Oral Surgery
  • Pathology and Forensic Medicine
  • Radiology Nuclear Medicine and imaging
  • Dentistry (miscellaneous)

Cite this

Interobserver agreement in dysplasia grading : Toward an enhanced gold standard for clinical pathology trials. / Speight, Paul M.; Abram, Timothy J.; Floriano, Pierre N.; James, Robert; Vick, Julie; Thornhill, Martin H.; Murdoch, Craig; Freeman, Christine; Hegarty, Anne M.; D'Apice, Katy; Kerr, A. Ross; Phelan, Joan; Corby, Patricia; Khouly, Ismael; Vigneswaran, Nadarajah; Bouquot, Jerry; Demian, Nagi M.; Weinstock, Y. Etan; Redding, Spencer W.; Rowan, Stephanie; Yeh, Chih Ko; McGuff, H. Stan; Miller, Frank R.; McDevitt, John.

In: Oral Surgery, Oral Medicine, Oral Pathology and Oral Radiology, Vol. 120, No. 4, 01.10.2015, p. 474-482.e2.

Research output: Contribution to journalArticle

Speight, PM, Abram, TJ, Floriano, PN, James, R, Vick, J, Thornhill, MH, Murdoch, C, Freeman, C, Hegarty, AM, D'Apice, K, Kerr, AR, Phelan, J, Corby, P, Khouly, I, Vigneswaran, N, Bouquot, J, Demian, NM, Weinstock, YE, Redding, SW, Rowan, S, Yeh, CK, McGuff, HS, Miller, FR & McDevitt, J 2015, 'Interobserver agreement in dysplasia grading: Toward an enhanced gold standard for clinical pathology trials', Oral Surgery, Oral Medicine, Oral Pathology and Oral Radiology, vol. 120, no. 4, pp. 474-482.e2. https://doi.org/10.1016/j.oooo.2015.05.023
Speight, Paul M. ; Abram, Timothy J. ; Floriano, Pierre N. ; James, Robert ; Vick, Julie ; Thornhill, Martin H. ; Murdoch, Craig ; Freeman, Christine ; Hegarty, Anne M. ; D'Apice, Katy ; Kerr, A. Ross ; Phelan, Joan ; Corby, Patricia ; Khouly, Ismael ; Vigneswaran, Nadarajah ; Bouquot, Jerry ; Demian, Nagi M. ; Weinstock, Y. Etan ; Redding, Spencer W. ; Rowan, Stephanie ; Yeh, Chih Ko ; McGuff, H. Stan ; Miller, Frank R. ; McDevitt, John. / Interobserver agreement in dysplasia grading : Toward an enhanced gold standard for clinical pathology trials. In: Oral Surgery, Oral Medicine, Oral Pathology and Oral Radiology. 2015 ; Vol. 120, No. 4. pp. 474-482.e2.
@article{d6b68fe7977044bdba7b1beaa8ffd646,
title = "Interobserver agreement in dysplasia grading: Toward an enhanced gold standard for clinical pathology trials",
abstract = "Objective. Interobserver agreement in the context of oral epithelial dysplasia (OED) grading has been notoriously unreliable and can impose barriers for developing new molecular markers and diagnostic technologies. This paper aimed to report the details of a 3-stage histopathology review and adjudication process with the goal of achieving a consensus histopathologic diagnosis of each biopsy. Study Design. Two adjacent serial histologic sections of oral lesions from 846 patients were independently scored by 2 different pathologists from a pool of 4. In instances where the original 2 pathologists disagreed, a third, independent adjudicating pathologist conducted a review of both sections. If a majority agreement was not achieved, the third stage involved a face-to-face consensus review. Results. Individual pathologist pair κ values ranged from 0.251 to 0.706 (fair-good) before the 3-stage review process. During the initial review phase, the 2 pathologists agreed on a diagnosis for 69.9{\%} of the cases. After the adjudication review by a third pathologist, an additional 22.8{\%} of cases were given a consensus diagnosis (agreement of 2 out of 3 pathologists). After the face-to-face review, the remaining 7.3{\%} of cases had a consensus diagnosis. Conclusions. The use of the defined protocol resulted in a substantial increase (30{\%}) in diagnostic agreement and has the potential to improve the level of agreement for establishing gold standards for studies based on histopathologic diagnosis.",
author = "Speight, {Paul M.} and Abram, {Timothy J.} and Floriano, {Pierre N.} and Robert James and Julie Vick and Thornhill, {Martin H.} and Craig Murdoch and Christine Freeman and Hegarty, {Anne M.} and Katy D'Apice and Kerr, {A. Ross} and Joan Phelan and Patricia Corby and Ismael Khouly and Nadarajah Vigneswaran and Jerry Bouquot and Demian, {Nagi M.} and Weinstock, {Y. Etan} and Redding, {Spencer W.} and Stephanie Rowan and Yeh, {Chih Ko} and McGuff, {H. Stan} and Miller, {Frank R.} and John McDevitt",
year = "2015",
month = "10",
day = "1",
doi = "10.1016/j.oooo.2015.05.023",
language = "English (US)",
volume = "120",
pages = "474--482.e2",
journal = "Oral Surgery, Oral Medicine, Oral Pathology and Oral Radiology",
issn = "2212-4403",
publisher = "Elsevier USA",
number = "4",

}

TY - JOUR

T1 - Interobserver agreement in dysplasia grading

T2 - Toward an enhanced gold standard for clinical pathology trials

AU - Speight, Paul M.

AU - Abram, Timothy J.

AU - Floriano, Pierre N.

AU - James, Robert

AU - Vick, Julie

AU - Thornhill, Martin H.

AU - Murdoch, Craig

AU - Freeman, Christine

AU - Hegarty, Anne M.

AU - D'Apice, Katy

AU - Kerr, A. Ross

AU - Phelan, Joan

AU - Corby, Patricia

AU - Khouly, Ismael

AU - Vigneswaran, Nadarajah

AU - Bouquot, Jerry

AU - Demian, Nagi M.

AU - Weinstock, Y. Etan

AU - Redding, Spencer W.

AU - Rowan, Stephanie

AU - Yeh, Chih Ko

AU - McGuff, H. Stan

AU - Miller, Frank R.

AU - McDevitt, John

PY - 2015/10/1

Y1 - 2015/10/1

N2 - Objective. Interobserver agreement in the context of oral epithelial dysplasia (OED) grading has been notoriously unreliable and can impose barriers for developing new molecular markers and diagnostic technologies. This paper aimed to report the details of a 3-stage histopathology review and adjudication process with the goal of achieving a consensus histopathologic diagnosis of each biopsy. Study Design. Two adjacent serial histologic sections of oral lesions from 846 patients were independently scored by 2 different pathologists from a pool of 4. In instances where the original 2 pathologists disagreed, a third, independent adjudicating pathologist conducted a review of both sections. If a majority agreement was not achieved, the third stage involved a face-to-face consensus review. Results. Individual pathologist pair κ values ranged from 0.251 to 0.706 (fair-good) before the 3-stage review process. During the initial review phase, the 2 pathologists agreed on a diagnosis for 69.9% of the cases. After the adjudication review by a third pathologist, an additional 22.8% of cases were given a consensus diagnosis (agreement of 2 out of 3 pathologists). After the face-to-face review, the remaining 7.3% of cases had a consensus diagnosis. Conclusions. The use of the defined protocol resulted in a substantial increase (30%) in diagnostic agreement and has the potential to improve the level of agreement for establishing gold standards for studies based on histopathologic diagnosis.

AB - Objective. Interobserver agreement in the context of oral epithelial dysplasia (OED) grading has been notoriously unreliable and can impose barriers for developing new molecular markers and diagnostic technologies. This paper aimed to report the details of a 3-stage histopathology review and adjudication process with the goal of achieving a consensus histopathologic diagnosis of each biopsy. Study Design. Two adjacent serial histologic sections of oral lesions from 846 patients were independently scored by 2 different pathologists from a pool of 4. In instances where the original 2 pathologists disagreed, a third, independent adjudicating pathologist conducted a review of both sections. If a majority agreement was not achieved, the third stage involved a face-to-face consensus review. Results. Individual pathologist pair κ values ranged from 0.251 to 0.706 (fair-good) before the 3-stage review process. During the initial review phase, the 2 pathologists agreed on a diagnosis for 69.9% of the cases. After the adjudication review by a third pathologist, an additional 22.8% of cases were given a consensus diagnosis (agreement of 2 out of 3 pathologists). After the face-to-face review, the remaining 7.3% of cases had a consensus diagnosis. Conclusions. The use of the defined protocol resulted in a substantial increase (30%) in diagnostic agreement and has the potential to improve the level of agreement for establishing gold standards for studies based on histopathologic diagnosis.

UR - http://www.scopus.com/inward/record.url?scp=84941600523&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84941600523&partnerID=8YFLogxK

U2 - 10.1016/j.oooo.2015.05.023

DO - 10.1016/j.oooo.2015.05.023

M3 - Article

VL - 120

SP - 474-482.e2

JO - Oral Surgery, Oral Medicine, Oral Pathology and Oral Radiology

JF - Oral Surgery, Oral Medicine, Oral Pathology and Oral Radiology

SN - 2212-4403

IS - 4

ER -