451 - Retrospective comparison of two modified screening tools for retinopathy of prematurity at a level IV neonatal intensive care unit in the United States

Publication Number: 451.6307

Shoji O. Samson, University of Illinois College of Medicine, Peoria, Peoria, IL, United States

Background: Retinopathy of prematurity (ROP), caused by abnormal retinal vascularization in premature infants, is a leading preventable cause of vision impairment worldwide. U.S. guidelines recommend screening all at-risk neonates, using high-sensitivity but low-specificity criteria that lead to frequent, resource-intensive exams, even for infants with mild or no ROP. Modified screening models incorporating postnatal weight gain have shown high sensitivity and specificity, enhancing detection and reducing unnecessary exams for lower-risk infants, thereby optimizing resource use.

Objective: The objective of this study is to compare the sensitivity and specificity of DIGIROP-Screen (ROP screening information and birth characteristics) vs G-ROP (postnatal growth and retinopathy of prematurity) using single institution cohort.

Design/Methods: This single-institution retrospective review included NICU infants born within OSF Healthcare System between 01/01/2017–12/31/2022 who met U.S. ROP screening criteria. Exclusions were neonates with non-physiological weight gain conditions, those not meeting criteria, or those lost to follow-up. Demographics, adverse events, ROP diagnoses, and treatment data were collected. The DIGIROP and G-ROP algorithms were compared to U.S. guidelines: DIGIROP uses birth GA, weight, gender, and exam dates, while G-ROP applies six sequential thresholds. Sensitivity, specificity, and 95% confidence intervals were calculated via the Wilson score, with McNemar’s test and confusion matrices used to assess accuracy.

Results: Of the 545 infants meeting initial screening criteria, 51 were excluded due to loss to follow-up. There were no significant demographic differences in the remaining cohort. The DIGIROP model showed moderate performance with 53.88% accuracy (95% CI: 49.01%–58.7%), a high sensitivity of 97.56% (effective in detecting true positives), but low specificity at 49.22%, misclassifying nearly half of true negatives. In contrast, the G-ROP model demonstrated 87.76% accuracy (95% CI: 84.27%–90.73%) and high specificity at 97.13%, yet had 0% sensitivity, failing to identify any true positives.

Conclusion(s): Comparing the two models, DIGIROP’s high sensitivity may be advantageous for maximizing true positive detection, despite lower specificity. Conversely, G-ROP’s high specificity but lack of sensitivity suggests it may better minimize false positives, though improvements are needed to capture true positives. Low McNemar's test p-values for DIGIROP indicate significant differences between observed and expected classifications, suggesting areas for model refinement to enhance performance.

Confusion Matrix for DIGI-ROP Screen
Performance metrics of the Digital ROP Screening model show moderate accuracy (53.88%, 95% CI: 49.01%–58.7%) with high sensitivity (97.56%) but low specificity (49.22%), indicating effective detection of true positives but poor exclusion of true negatives. The low positive predictive value (17.02%) further suggests a high rate of false positives, potentially limiting the model’s clinical utility where specificity is critical.

Confusion Matrix for G-ROP Model
The G-ROP model exhibits high accuracy (87.76%, 95% CI: 84.27%–90.73%) and specificity (97.13%), effectively identifying true negatives. However, its sensitivity is 0%, indicating a failure to detect any true positive cases, which is critical for a model intended to identify conditions requiring intervention. This conservative approach minimizes false positives but significantly limits the model’s effectiveness in detecting true positives.