Local Validation of G-ROP and Modified G-ROP Criteria in the Detection of Prethreshold Retinopathy of Prematurity

Retinopathy of prematurity (ROP) is a potentially blinding, retinal vasoproliferative disorder in premature infants. While majority of ROP resolves without permanent damage to the eye, approximately 10% of infants with ROP develop sight-threatening complications.¹ A local study by Del Mundo reported that ROP is the leading treatable cause of blindness among children enrolled in a school for the blind and highlighted the importance of screening at-risk infants.²

In 2013, the Philippine Academy of Ophthalmology Retinopathy of Prematurity Working Group (PAO-ROPWG) published a screening criteria for ROP which included all premature infants with gestational age (GA) less than 35 weeks, birth weight (BW) less than 2,000 grams, or infants with GA > 35 weeks or BW > 2000 grams and unstable clinical course.³ In 2020, the PAO-ROPWG issued an updated consensus statement revising the screening criteria for ROP (Figure 1).⁴ In the revised criteria, the GA and BW cut-offs were set low to ensure that all at-risk infants and those requiring treatment were examined. However, it subjected infants who are at low-risk of developing ROP to undergo an unnecessary screening. Indeed, there is still room to improve the ROP screening criteria to decrease the burden on the health system especially in a resource-limited setting like the Philippines.⁵

Studies have shown the association of low serum insulin-like growth factor 1 (IGF-1) levels with increased risk of ROP.⁶ Post-natal weight gain, a surrogate for serum IGF-1, is a useful screening criterion that has been incorporated into several ROP screening models. Most recently, a new screening criteria from the postnatal growth and ROP (G-ROP) study has been reported to achieve 100% sensitivity and 35.6% specificity in predicting type 1 ROP compared to the current standard ROP screening criteria (Figure 2).⁷ Applying the G-ROP criteria to their cohort of infants, the investigators were able to capture all infants who developed type 1 ROP as well as identify 30% of infants who did not need ROP examination.⁷ A key point of the G-ROP criteria is that it should not be broadly applied in countries where ROP development is primarily driven by excessive oxygen use.

Locally, the PAO-ROPWG identified the use of oxygen supplementation as one of the criteria for screening older and heavier infants.⁴ The modified G-ROP criteria was developed by the investigators to take into consideration supplemental oxygen exposure longer than 24 hours on top of the published G-ROP screening criteria. This modification to the G-ROP criteria aimed to improve its sensitivity and specificity and make it more applicable for the Filipino demographic (Figure 3).

This study determined and compared the diagnostic accuracies of G-ROP and modified GROP in predicting infants who developed ROP in a 10-year cohort in a single tertiary hospital. Specifically, the study determined the sensitivity, specificity, positive and negative predictive values of G-ROP and modified G-ROP in predicting development of prethreshold (type 1 or type 2) ROP, and identifying low risk infants.

METHODS

This study was a single-center, cross-sectional, retrospective study that included infants referred for ROP screening in a tertiary hospital. Ethical clearance was granted by the Institutional Ethics Review Committee. Medical records of infants admitted in the neonatal intensive care unit (NICU) who underwent ROP screening from January 2012 to December 2021 were reviewed. Only infants with known ROP outcomes were included in the study. Infants with incomplete medical records or those screened as outpatient were excluded from the study. Birth weight (BW) and gestational age (GA) cut-offs were not used as inclusion criteria in order to make the cohort fully representative of all infants undergoing ROP examinations. All ROP screening was performed by either a retina or pediatric ophthalmology specialist after pharmacologic pupil dilation. The type of ROP was recorded based on the Early Treatment for Retinopathy of Prematurity.⁸

Type 1 ROP included a diagnosis of zone I, any stage ROP with plus disease; zone I, stage 3 ROP without plus disease; zone II, stage 2 or 3 with plus disease; and aggressive posterior ROP. Type 2 ROP are those diagnosed with ROP which is limited to zone I, stage 1 or 2 without plus disease, or zone II, stage 3 without plus disease. ⁶ Mild ROP are those diagnosed with ROP not included in the type 1 and type 2 ROP definition. These are mild retinal changes outside the range of normal development but not severe enough to meet criteria for type 2 ROP.

Demographic and clinical data, including gender, BW, GA, daily weight gain, ROP stage in the worse eye, and type and duration of oxygen supplementation, were collected from the medical records. In infants who received more than 1 type of oxygen supplementation, all forms of oxygen delivery were recorded and counted.

Since the infants included in the study were screened from 2012 to 2021, the previous 2013 PAO-ROPWG criteria were mainly used as screening criteria for ROP referral. The 2020 PAO-ROPWG screening consensus, G-ROP, and modified G-ROP criteria were applied after reviewing the medical records of infants referred for ROP screening. An infant required examination using the 2020 PAOROPWG screening consensus if he/she met any of the criteria in Figure 1, otherwise, the infant was labelled as low risk. An infant was labelled as requiring examination if he/she met 1 or more of the quantitative thresholds seen in Figure 2 for G-ROP criteria and Figure 3 for modified G-ROP criteria. Those who did not meet any of the G-ROP or modified G-ROP criteria stated were labeled as lowrisk infants. ROP outcomes of all infants were reviewed to determine the number of correctly predicted ROP. The proportion of low risk infants were calculated from infants labelled as not requiring examination using the 2020 PAO-ROPWG consensus, G-ROP or modified G-ROP divided by the total number of infants referred for screening.

The primary outcomes were sensitivity rates for predicting prethreshold and mild ROP using the GROP and modified G-ROP criteria. Secondary outcomes include specificity rates, positive predictive values (PPV), and negative predictive values (NPV) for predicting prethreshold ROP using the G-ROP and modified G-ROP, and percentage of infants who are low risk.

Statistical Analysis

Measures of central tendencies and dispersion were used to describe the demographic and clinical characteristics. The 95% confidence interval for sensitivity and specificity were calculated using the Wilson method. One-way ANOVA with post hoc test for BW and AOG was performed per ROP outcome. Chi-square test was used to compare the G-ROP, modified G-ROP criteria and currently recommended consensus by the PAO-ROPWG. Chi-square test was used to compare low risk infants (%) between the G-ROP, modified G-ROP criteria and current PAO-ROPWG consensus. All statistical analyses were performed using Stata version 14 (Statacorp LLC, College Station, Texas, USA).

RESULTS 

There were 969 infants referred for ROP screening from January 2012 to December 2021. Ninety-six (96) infants had no known ROP outcome. A total of 873 infants was included in the analysis. Table 1 summarizes the demographic and clinical characteristics of infants included in the study. There were 448 males (51.4%) and 425 females (48.6%). The mean GA was 32.8 ± 2.8 weeks (range, 23-40 weeks). The mean BW was 1,818.7 ± 581. 2 grams (range, 500-4,022 grams). Most infants did not receive supplemental oxygen (57.6%). Among those who received supplemental oxygen, low flow cannula (30.9%) was the most common method of delivery followed by high flow cannula (16.2%) and continuous positive airway pressure (15.3%). There were 226 infants who received more than 1 form of oxygen support.

Table 2 shows 162 (18.6%) infants developed ROP. Of these, 25 (2.7%) developed type 1 ROP, 27 (3.1%) developed type 2 ROP, and 110 (12.6%) developed mild ROP. One-way ANOVA with post hoc test for BW and AOG analysis showed significant differences (P=0.001) on the development of ROP in terms of mean BW and GA. Specifically, mean BW of infants with type 1 ROP was significantly lower than infants with mild ROP and no ROP (863.6, 1271.2, and 1,953.3 grams, respectively) [p=0.001]. Likewise, the mean BW of 948.6 grams of infants with type 2 ROP was significantly lower than infants with mild ROP and no ROP [p=0.001]. The mean GA of infants with type 1 ROP was significantly lower than infants with mild ROP and no ROP (26.8, 30.0 and 33.7 weeks, respectively) [p=0.001]. Likewise, infants with type 2 ROP had significantly lower GA of 27.6 weeks compared to mild ROP and no ROP [p=0.001]. Moreover, the means of BW and GA of infants with type 1 and 2 ROP were not significantly different.

Among infants who developed type 1 or type 2 ROP, the most common form of oxygen supplementation was through mechanical ventilation. Low flow cannula was the most common form of oxygen supplementation among those with mild ROP or no ROP.

Using the G-ROP criteria, all infants with type 1 ROP (25 of 25) were correctly predicted giving a sensitivity rate of 100% (95% CI: 86.3-100.0%) (Table 3). In addition, G-ROP criteria were able to correctly predict 24 out of 27 infants with type 2 ROP, with a sensitivity rate of 88.9% (95% CI: 70.8- 97.6%). Among infants with mild ROP, the sensitivity rate of G-ROP was 64.5% (95% CI: 54.9- 73.4%).

Using the modified G-ROP criteria, all 25 infants with type 1 ROP and all 27 infants with type 2 ROP were correctly predicted, with sensitivity rates of 100% (95% CI: 86.3-100%) and 100% (95% CI: 87.2-100%), respectively (Table 4). Majority of infants with mild ROP were predicted using G-ROP (sensitivity, 93.6%; 95% CI: 87.3% to 97.4%).

The specificity of G-ROP and modified G-ROP criteria in predicting type 1 ROP was 79.2% (95% CI, 76.4% – 81.9%) and 54.9% (95% CI, 51.5% – 58.3%), respectively. These rates were higher compared to PAO-ROPWG consensus which had a specificity of 39.9% (95% CI, 36.5% – 43.2%). Likewise, specificity of G-ROP and modified G-ROP criteria in predicting type 2 ROP was 79.1% (95% CI, 76.2% – 81.8%) and 55.1% (95% CI, 51.7% – 58.5%), respectively. These were again higher compared to PAO-ROPWG consensus which had a specificity of 39.9% (95% CI, 36.6% – 43.3%).

For type 1 ROP, the PPVs for G-ROP, modified G-ROP and PAO-ROPWG criteria were PPV of 12.4% (95% CI: 11.1%-13.9%), 6.14% (95% CI: 5.7% – 6.6%) and 4.67% (95% CI: 4.4% – 4.9%), respectively . The NPVs of G-ROP, modified GROP criteria and PAO-ROPWG consensus were 100% each. For type 2 ROP, The PPVs for G-ROP, modified G-ROP, and PAO-ROPWG criteria were 11.9% (95% CI: 10.1%-14.1%), 6.6% (95% CI: 6.2%- 7.1%), and 5.1% (95% CI: 4.8%-5.3%) respectively. The NPV for modified G-ROP criteria and PAOROPWG were 100% which was slightly higher than the G-ROP criteria 99.6% (95% CIL 98.7% – 99.9%) (Table 5).

Table 6 shows that percentage of identifying infants who are low-risk. PAO-ROPWG consensus, G-ROP, and modified GROP identified 37.0, 76.6 and 54.1% of infants who are low risk for ROP.

DISCUSSION

This study validated the G-ROP and modified GROP diagnostic criteria in a cohort of 873 infants seen over a 10-year period in a single institution. Our study findings showed reproducible performance of both ROP screening criteria with 100% sensitivity in detecting type 1 ROP which is comparable with a previous international study.⁷ The sensitivity of the G-ROP and modified G-ROP criteria in detecting type 1 ROP were comparable with the 2020 PAOROPWG consensus in terms of population for ROP screening. Majority of the infants who developed type 1 ROP had very low BW (<1010 grams) and GA (<28 weeks) which were examined at 31 weeks GA. Four (4) infants who developed type 1 ROP were successfully identified using the G-ROP and modified G-ROP criteria despite not having met the criteria of BW and GA but where captured due to slow post-natal weight gain during the first 10-19 days of life. Initial screening were performed at 20 days chronologic age for these infants. This may imply the utility of post-natal weight gain as a surrogate marker for infants developing prethreshold ROP in less at risk infants or those born with higher GA and BW.

The G-ROP criteria missed 3 (11%) infants who developed type 2 ROP. Binenbaum et al. also reported a similar finding in their US cohort wherein 1.2% of infants who developed type 2 ROP were missed when the G-ROP criteria was used.⁷ In these infants, other risk factors for the development of ROP should be assessed. Interestingly, in our cohort, the infants who developed type 2 ROP missed by the G-ROP criteria were detected using the modified GROP criteria. These infants had higher BW and GA and good post-natal weight gain but were only captured due to a positive history of oxygen supplementation. Although ROP regressed spontaneously in all these infants and treatment is not currently recommended for eyes with type 2 ROP, this highlights the need to include infants exposed to oxygen supplementation for 24 hours or more as this provides an added layer of screening. Locally, due to the unavailability of oxygen blenders, pure oxygen is being delivered to infants which contributes to a hyperoxic state, a known risk factor for the development of ROP. Determining the type of oxygen exposure, whether given by oxygen mask or mechanical ventilation, and duration of oxygen supplementation can further streamline the modified G-ROP criteria for future studies.

Specificity rates and positive predictive values for G-ROP and modified G-ROP criteria were significantly higher (P= 0.0001) than the current ROP consensus (Table 5). This is due to the stricter criteria (i.e., lower BW and GA thresholds) compared with our local guidelines.⁴ While still achieving 100% sensitivity in detecting prethreshold ROP, this might imply that in this cohort of infants, stricter BW and GA with postnatal weight gain monitoring may aid in predicting infants requiring examination. The utility of the post-natal weight gain provides another safety net in detecting infants at risk for developing type 1 ROP who passed the initial criteria of GA <28 weeks and BW <1,051 grams. The PAO-ROPWG consensus has a more lax screening criteria (i.e. BW < 32 weeks and BW < 1,500) compared to the GROP criteria. This study looked at the window of GA between 28 to 32 weeks and BW between 1,051 to 1,500 grams, and whether factoring in post-natal weight gain and oxygen supplementation can detect those who will develop ROP.

The G-ROP and modified G-ROP criteria demonstrated similar sensitivity with the current ROP screening consensus in predicting infants who will develop type 1 ROP with the benefit of better identifying low risk infants (77% using G-ROP criteria vs 54% using modified G-ROP criteria) (Table 5). Compared to Binenbaum et al., G-ROP decreased the numbers needed for screening to 35.2% which was mainly due to difference in the BW (<1,051 grams) and GA (<28 weeks) criteria compared to their local criteria for screening (BW <1,500 grams or GA < 30 weeks).⁷ These infants who are low risk may not need to undergo initial screening or may receive fewer examinations with longer intervals. In our study, the percentage of low risk infants were only counted based on the known ROP diagnosis, hence an infant who had 3 ROP examinations will only count as 1 outcome. Therefore, identifying low risk infants not only decreases the initial screening but also the subsequent examinations which may potentially be more costeffective by allocating resources to high risk infants. These potential application of the G-ROP and modified G-ROP criteria need to be further analyzed for clinical use locally.

The G-ROP and modified G-ROP criteria are both easy to use which facilitates their practical application. If the infant met either the first 2 criteria of low BW or GA, then postnatal weight gain need not be taken into account. As for the rest of the infants, body weight is routinely measured on a daily basis in the neonatal intensive care unit and this data is readily available. Once an infant is identified to have a slow post-natal weight gain or given oxygen supplementation for more than 24 hours, only then is screening performed. The high sensitivity and specificity in detecting prethreshold ROP of G-ROP and modified G-ROP criteria, in addition to identifying low risk infants may help decrease the burden on ophthalmologists, as well as the healthcare system, and make them potential ROP screening tools.

Limitations of this study include generalizability of the results from infants confined at a single institution. Applying the criteria to a wider population, and including highly vulnerable infants can verify their reproducibility and reliability. G-ROP criteria should be applied with caution in countries with excessive oxygen supplementation, hence the development of modified G-ROP criteria. The timing of initial ROP screening in the United States also differs locally. Future studies on modified GROP criteria may standardize the timing of initial screening using the 2020 PAO-ROPWG consensus. A prospective analysis can be performed by using the G-ROP and modified G-ROP criteria as well as considering the other risk factors in the current PAO-ROPWG consensus. Additionally, it would be insightful to determine whether ROP outcomes are worse in infants who meet the criteria for low BW and GA but experience slow post-natal weight gain, compared to those with typical post-natal weight gain.

Oxygen supplementation still plays an important role in the development of ROP especially in areas where a highly developed neonatal care system is lacking. In this study, only the presence or absence of oxygen supplementation was considered. The type and duration of oxygen delivery, and infant’s oxygen levels were not analyzed and should be further investigated.

Another recommendation is the development of a digital application to aid in monitoring infants for ROP screening. This digital app would include the GROP and modified G-ROP criteria for GA, BW, daily weight gain, and oxygen exposure. When any of these criteria are met, the app would alert the neonatologist, indicating that ROP screening should be considered for the infant. The timing of the ROP examination will then depend on the 2020 PAOROPWG recommendation.

In summary, the G-ROP and modified G-ROP criteria, compared to the PAO-ROPWG consensus, yielded equal sensitivity rates and higher specificity rates in detecting the presence of prethreshold ROP in a cohort of infants. This high sensitivity rate coupled with a clinical potential to identify more low risk infants may improve the efficiency of ROP screening in this tertiary center. Generalizability of G-ROP and modified G-ROP to other institutions and possibility of incorporating into national ROP guidelines need further investigation.

REFERENCES

1. Bautista G, Ventura, R. Local Validation of WINROP, an Online Screening Tool for Retinopathy of Prematurity. Philipp J Ophthalmol 2018;43:15-18.
2. Del Mundo P, Chua C. Causes of blindness and severe visual impairment among children enrolled in an early intervention and preschool program of a school for the blind in the Philippines. Philipp J Ophthalmol. 2015;40:41-46.
3. Philippine Academy of Ophthalmology – ROP Working Group (PAO-ROPWG). Recommended Philippine Guidelines for Screening and Referral of Retinopathy of Prematurity (ROP). November 17, 2023: https://vrsp.org.ph/wpcontent/uploads/2018/08/2013-ROP-GUIDELINESver.112613.pdf (accessed July 26, 2024).
4. Philippine Academy of Ophthalmology – ROP Working Group (PAO-ROPWG), Philippine Society of Newborn Medicine (PSNbM), Philippine Pediatric Society (PPS). Retinopathy of Prematurity Philippine Preventive Care Plan Strategy. 2020: https://vrsp.org.ph/wpcontent/uploads/2022/05/1589017296586941.pdf (accessed January 15, 2022).
5. Naguib M, Soares R, Anzures, R, et al. Regionally Specific Economic Impact of Screening and Treating Retinopathy of Prematurity in Middle-Income Societies in the Philippines. J Pediatr Ophthalmol Strabismus 2019;56:388- 396.
6. Hellstrom A, Perruzzi C, Ju M, et al. Low IGF-I suppresses VEGF-survival signaling in retinal endothelial cells: direct correlation with clinical retinopathy of prematurity. Proc Natl Acad Sci USA. 2001;98(10):5804-5808.
7. Binenbaum G, Tomlinson LA, de Alba Campomanes AG, et al Postnatal Growth and Retinopathy of Prematurity (GROP) Study Group. Validation of the Postnatal Growth and Retinopathy of Prematurity Screening Criteria. JAMA Ophthalmol. 2020;138(1):31-37.
8. Good WV; Early Treatment for Retinopathy of Prematurity Cooperative Group. The Early Treatment for Retinopathy Of Prematurity Study: structural findings at age 2 years. Br J Ophthalmol 2006;90(11):1378-82