Defining and Assessing the Reproducibility of Crohn's Disease Endoscopic Lesions: A Delphi-like Method from the GETAID

Background and Aims: Defining and assessing the reproducibility of Crohn's disease [CD] endoscopic lesions is essential in assessing endoscopic healing. Methods: Twelve endoscopic CD experts from the GETAID defined aphthoid erosions [AE], superficial ulcerations [SU], deep ulcerations [DU], stenosis, and fistulas according to a Delphilike method. Thirty different GETAID physicians declared if they found acceptable each definition. Intra- and inter-observer agreements were investigated using 100 videos with one tagged specific lesion [AE, SU, DU, or sham lesion] read by 15 independent endoscopists at baseline and 1 month later in a randomised order. Video quality was determined by an external reader. According to kappa estimate [kappa +/- standard error], intra or inter-observer agreement was qualified as `moderate' [0.4-0.6], `substantial' [0.6-0.8], or `almost perfect' [0.8-1.0]. Results: Among 30 different experts, 83% to 97% found acceptable the definitions retrieved from the Delphi-like method. Intra-observer kappa was 0.717 [+/- 0.019] for SU, 0.681 [+/- 0.027] for AE, 0.856 1 +/- 0.014] for DU, showing `substantial' agreement. It was 0.801 [+/- 0.016] for any ulceration [DU or SU]. There was a high variability across readers from `moderate' to `almost perfect' agreement. observer kappa was 0.548 [+/- 0.042] for SU, 0.554 [+/- 0.028] for AE 0.694 [+/- 0.041] for DU, and 0.705 [+/- 0.042] for any ulceration. Inter-observer agreement increased when reading the 53 high-quality videos: 0.787 [+/- 0.064] [p = 0.001], 0.607 [+/- 0.043] [p = 0.001], and 0.782 [+/- 0.064][p = 0.001] for DU, AE, and any ulceration, respectively. Conclusions: Despite variable intra-agreement level across readers, the GETAID definitions for CD endoscopic lesions provided `substantial' inter-observer agreements, especially in case of high-quality videos.