Local Validation of WINROP, an Online Screening Tool for Retinopathy of Prematurity

Retinopathy of prematurity (ROP) remains to be a common cause of preventable blindness in children all over the world.1 Sight-threatening disease develops in approximately 10-15% of babies with ROP. Early detection of preterm infants at risk for ROP through timely screening cannot be overemphasized.

While prematurity and low birth weight are widely-accepted risk factors for developing ROP, poor postnatal weight gain has been implicated in the development and progression of ROP. An online monitoring tool, WINROP (Weight, Insulin-like growth factor, Neonatal ROP) was recently developed to identify infants at high-risk for developing ROP based on weekly postnatal weight measurements. The web-based software calculates a preterm infant’s risk for developing any-stage or sight-threatening ROP based on his/her actual weekly weight gain. When the calculated risk exceeds the high-risk level, an alarm is triggered by the software.

Initial studies have reported varying sensitivity rates of the WINROP tool in detecting high-risk ROP infants, ranging from 50% to 100%. Given the wide variances in study results, it is important to conduct a local validation of the WINROP online tool. The objective of this study is to validate the WINROP online tool in predicting the development of any-stage ROP and treatment-requiring ROP among Filipino preterm infants in a single institution.

 

METHODOLOGY

Medical records of preterm infants admitted at the neonatal intensive care unit (ICU) of the East Avenue Medical Center, who underwent ROP screening, from January 2013 to April 2017 were reviewed. Patient confidentiality was strictly maintained in the course of the study with adherence to the standards set forth by the Declaration of Helsinki and ICH-GCP. Charts of preterm infants born with gestational age of 32 weeks or less and who were admitted at the neonatal ICU for at least 2 weeks were included. Charts that were incomplete or belonging to infants who died during the course of ROP screening or who did not complete the ROP screening due to other reasons, or who had non-physiologic weight gain (i.e. hydrocephalus) were excluded.

Screening, diagnosis, and treatment of ROP were performed by a pediatric ophthalmology fellow; Ophthalmologyfindings and diagnoses were confirmed by a pediatric ophthalmology consultant. ROP classification was based on the International Committee on the Classification of Acute ROP. Treatment was carried out based on the criteria of the Early Treatment for Retinopathy of Prematurity (ETROP) study which included zone I, any stage ROP with plus disease; zone I, stage 3 ROP without plus disease; zone II, stage 2 or 3 with plus disease; and aggressive posterior ROP. Weights obtained from the charts were measured using a digital scale.

Anonymized data such as sex, gestational age at birth, birth weight, weekly weight, most severe ROP stage in the worse eye, and treatment type, if done, were collected and entered on the free online screening tool WINROP. Weekly weights were entered in the online tool until 36 weeks gestational age, as recommended by the WINROP study.

WINROP results were then compared with the actual ROP diagnoses derived from the medical chart. The sensitivity and specificity rates of WINROP to detect ROP were calculated. In addition, the positive predictive values (PPV) and negative predictive values (NPV) were also calculated. Stata v13 were used in data processing and statistical analysis. A P-value less than 0.05 was considered significant.

 

RESULTS

Medical records of 138 preterm infants were included in the study. Half of preterm infants were male (51.4%). The median gestational age was 31 weeks (range: 26 to 32 weeks). Median birth weight was 1,285 grams (range: 800 to 1,935 grams) (Table 1). Sixty-four (64) of 138 (46.4%) of preterm infants had ROP. Nine infants (14.1%) had ROP type I, while 51 (79.7%) had ROP type II. Aggressive posterior retinopathy of prematurity (APROP) was observed in 4 (6.2%) preterm newborns.

Agreement Between WINROP Results and Chart Diagnosis
The agreement between WINROP results and actual chart diagnosis is plotted in Table 2. WINROP flagged 77 preterm infants (55.8%). Of the 77 infants who were tagged high-risk for ROP, 50 had ROP.

Table 1: Demographic Characteristics of Preterm Infants

SD: standard deviation, IQR: interquartile range

Table 2: Agreement Between WINROP and Chart Diagnosis on Any-stage ROP

 

Sixty-one (61) infants did not trigger an alarm. Out of which, 47 did not have ROP. Calculated PPV and NPV were 64.9% and 77.0%, respectively (Table 3).

The sensitivity and specificity rates of WINROP to detect any-stage ROP is also plotted in Table 3. Out of the 64 infants who had a chart diagnosis of ROP, 50 were flagged correctly by WINROP to be high-risk for ROP. Sensitivity of WINROP tool in detection of ROP was 78.1%. Of the 74 infants who did not develop ROP, 47 were not flagged by WINROP. Calculated specificity is 63.5%.

Table 3: Sensitivity, Specificity, Positive and Negative Predictive Values of WINROP in Predicting Any ROP

Table 4 shows the agreement between WINROP results and chart diagnosis based on treatment-requiring ROP. Seventy-seven (77) infants were flagged by WINROP to have treatment-requiring ROP. Only 10 of these 77 infants had treatment-requiring ROP from chart review. WINROP did not tag 60 infants with treatment-requiring ROP. Fifty-seven (57) of whom truly did not have treatment-requiring disease by chart review. The PPV and NPV values of WINROP for detecting treatment-requiring ROP were 13% and 95.1%, respectively (Table 5).

Table 4: Agreement Between WINROP and Chart Diagnosis on Treatment-requiring ROP

TR-ROP: treatment-requiring ROP

Table 5: Sensitivity, Specificity, Positive and Negative Predictive Values of WINROP in Predicting Treatment-requiring ROP

TR-ROP: treatment-requiring ROP

Among the 13 infants who truly had treatment-requiring ROP, 10 were correctly identified by WINROP. Sensitivity of WINROP in detecting treatment-requiring ROP in this cohort was 76.9%. Out of 125 who did not have treatment-requiring ROP, 58 preterm infants were not flagged correctly by WINROP. Specificity was 46.4%.

Lastly, among preterm infants with treatment-requiring ROP, the mean advanced alarm time, or interval between WINROP alarm to actual development of treatment-requiring ROP, was 5 weeks.

DISCUSSION
In this study, the online ROP surveillance system, WINROP, had fair sensitivity and specificity (78.1 and 63.5%, respectively) in predicting any-stage ROP among preterm infants from a single institution. Additionally, the sensitivity and specificity rates in detecting treatment-requiring ROP were 76.9% and 46.4%, respectively. These numbers are lower compared to previous studies done in other countries.4,7-9 Discrepancies in the results may be due to the presence of other postnatal factors in this cohort of premature infants. WINROP only takes into account postnatal weight gain when in fact ROP development and progression is multifactorial. Other postnatal factors including sepsis, respiratory distress syndrome, hyperbilirubinemia may contribute to development and progression of ROP.

WINROP is a free, online tool. However, the poor to fair sensitivity and specificity rates from this study limit its application in the local setting. WINROP may aid ROP prediction, but regular screening of preterm infants at risk for ROP based on current criteria remains to be the standard of care.

 

REFERENCES

1. Gilbert C. Retinopathy of prematurity: a global perspective of the epidemics, population of babies at risk and implications for control. Early Hum Dev. 2008;84:77-82.
2. The American Academy of Ophthalmology. 2014-2015 Basic and Clinical Science Course. Section 12: Retina and Vitreous. San Francisco, CA. 2014. 163 p.
3. pao.org.ph [internet]. RETINOPATHY OF PREMATURITY. ROP Working Group, Philippine Academy of Ophthalmology, c2014 (cited 2017 October) Available from: http://www.pao.org.ph/news_home/2014/rop/ROP% 20for%20 media%20_July%2028.pdf
4. Hård AL, Löfqvist C, Fortes Filho JB, et al. Predicting proliferative retinopathy in a Brazilian population of preterm infants with the screening algorithm WINROP. Arch Ophthalmol. 2010;128:1432-6.
5. Mitchell AJ, Green A, Jeffs DA, et al. Physiologic effects of retinopathy of prematurity screening examinations. Adv Neonatal Care. 2011;11:291-7.
6. Lundgren P, Stoltz Sjöström E, Domellöf M, et al. WINROP identifies severe retinopathy of prematurity at an early stage in a nation-based cohort of extremely preterm infants. PLoS ONE. 2013;8:e73256.
7. Wu C, Vanderveen DK, Hellström A, et al. Longitudinal postnatal weight measurements for the prediction of retinopathy of prematurity. Arch Ophthalmol. 2010;128:443-7.
8. Choi JH, Löfqvist C, Hellström A, et al. Efficacy of the screening algorithm WINROP in a Korean population of preterm infants. JAMA Ophthalmol. 2013;131:62-6.
9. Zepeda-Romero LC, Hård AL, Gomez-Ruiz LM, et al. Prediction of retinopathy of prematurity using the screening algorithm WINROP in a Mexican population of preterm infants. Arch Ophthalmol. 2012;130:720-3.
10. Nagaraj, K. Retinopathy of prematurity (ROP) in India and WINROP software. J Clin Exp Ophthalmol 2016 Jun (cited 2017 August 7). Available from http://pediatricophthalmology.conferenceseries.com/abstract/2016/retinopathy-of-prematurity-rop-in-india-and-winrop-software
11. The Sahlgrenska Center for Pediatric Ophthalmology Research. Prediction/WINROP; c2014 (cited 2017 August 7) Available from http://rop.gu.se/Prediction
12. National Eye Institute [internet]. Facts About Retinopathy of Prematurity (ROP); c2014 (cited 2017 September 12). Available from: https://nei.nih.gov/health/rop
13. Lundgren P, Stoltz Sjöström E, Domellöf M, et al. The specificity of the WINROP algorithm can be significantly increased by reassessment of the WINROP alarm. Neonatology. 2015;108:152-6.
14. Lee J, Dammann O. Perinatal infection, inflammation, and retinopathy of prematurity. Semin Fetal Neonatal Med. 2012;17:26-9.