A Qualitative Analysis of Challenges and Successes in Retinopathy of Prematurity Screening

Abstract

Objective The objective of this study is to identify characteristics of neonatal intensive care unit (NICU) practice that influence successful retinopathy of prematurity (ROP) screening.

Study Design In this qualitative study, top, improved, and bottom performing NICUs in the California Perinatal Quality Care Collaborative were identified based on ROP screening rates and invited to participate. NICU personnel were interviewed using a semistructured questionnaire. Using thematic analysis, key factors that influence ROP screening were identified.

Results Themes found in top performing hospitals include a commitment to quality improvement, a committed ophthalmologist, and a system of double checks. Improved NICUs had a common theme of utilizing telemedicine for exams and identification of eligible neonates on admission. The bottom performing hospital struggled with education and identification of eligible neonates and a lack of a dedicated ophthalmologist.

Conclusion Structure, culture, education, and commitment all contribute to the success of ROP screening in the NICU.

Keywords

ROP - quality improvement - qualitative research - process improvement

Retinopathy of prematurity (ROP) is a vascular disease of the eye, primarily affecting premature neonates. Despite our improved understanding of the disease and the screening guidelines to identify at-risk neonates, ROP remains a leading cause of blindness in the United States.[1] Twenty-five years after the CRYO-ROP study demonstrated benefit to treatment of infants with ROP, studies show that not all infants are being screened appropriately for ROP.[2] A previous study demonstrated that the median rate of missed ROP screens for eligible infants in California was 13%, with some hospitals missing up to 73% of infants who qualified.[2]

As more premature babies are surviving, there is more demand for ROP evaluations. A recent study showed that the incidence of ROP increased from 14.7% in 2000 to 19.9% in 2012.[3] It is thus more important than ever to understand why eligible infants are not being identified and screened for ROP.

In this qualitative study, we aimed to understand what barriers exist to identifying and screening neonates for ROP and what systems may help to overcome these barriers. Identifying these factors will provide the foundation for quality improvement (QI) activities to optimize practice and promote better outcomes.

Subjects and Methods

We interviewed neonatal intensive care unit (NICU) hospital staff at six hospitals in California between December 2012 and November 2015. Sites were identified by their ROP screening rates based on individual patient level data as collected by the California Perinatal Quality Care Collaborative (CPQCC). The CPQCC collects data in a systematic fashion using standardized definitions developed by the Vermont Oxford Network. Screening rates of all NICUs in California were analyzed anonymously, and evaluated via crude and adjusted ROP screening rates. Among the top 10 and bottom 10 performers out of approximately 130 NICUs, we selected several geographically diverse units of various sizes to have a representative group.

Sites were contacted through email with their medical director. Semistructured interviews were conducted in person and by phone. An interview guide (Appendix A) was used to guide conversations; however, we allowed discussions to stray from the guide at times. In-person interviews were done both individually, and in groups of 2 to 4 people and were grouped by role in the NICU. At each hospital, we spoke to NICU staff members including neonatologists, bedside nurses, discharge coordinators, social workers, and ophthalmologists.

Interviews were audio recorded, transcribed by a professional transcriptionist, and deidentified. Data were independently coded and analyzed by two researchers manually using established grounded theory methods.[4] Theory was generated through the constant comparison method, where new observations are constantly compared with previously collected data and categories are continually developed.[5] Initial codes were identified by line-by-line coding. Analytic memos were written to detail emerging categories, ideas, and concepts. The investigators discussed and resolved all discrepancies. Findings were synthesized into major themes and a conceptual model. Saturation was reached when no new themes were emerging with subsequent transcription analysis.

Results

Screening rates of all NICUs in California were analyzed anonymously, and evaluated via crude and adjusted ROP screening rates. Sites visited represented variety in location (urban vs. rural), volume, and Clinical Classifications Software level. Three hospitals were identified as high performing, two were initially low performers but demonstrated temporal improvement (from 2010–2011 to 2012–2013), and two were identified as a low performing hospitals. We received responses from all NICUs contacted except for one low performer.

Based on the interview, four major themes were identified as common to high performing hospitals, three for improved hospitals, and three for the low performing hospital. A conceptual model summarizing the findings is shown in [Fig. 1]. Motivating themes and barriers were also identified for ophthalmologists.

Fig. 1 Synthesis of key factors for successful retinopathy of prematurity (ROP) screening.

Top Performing Hospitals

Themes common to top performing hospitals include an overall commitment to QI and participation in QI projects, a committed ophthalmologist, and a system with double checks and reminders.

Overall commitment to QI includes active participation in statewide collaboratives, frequent ongoing QI projects led by various staff members, and a sense from all staff that the unit is constantly being evaluated to identify opportunities for improvement. This theme was noted when multiple members of a NICU in various roles independently identified ongoing QI as a goal of the NICU.

Dedication of a committed ophthalmologist was noted when a unit denied having difficulty obtaining ROP screens for their infants. These units describe one or more ophthalmologists who routinely come to the unit to examine infants, and find coverage for times that they are unavailable. They acknowledge that they are in the minority of units and are thankful that the ophthalmologists who provide their services are committed to the babies.

A system with double checks and reminders was also a theme of successful NICUs. In these systems, no one person is responsible for identifying neonates who are eligible for screening. There are often several people from the desk clerk to the social worker to the nurse and physician who all work to identify neonates. Reminders are often part of note templates, and there is often also a central log in a binder or on a white board that lists eligible infants and the date the exam is due.

Improved Hospitals

Themes common to improved hospitals include identification of eligible neonates on admission, the use of Retcam (a telemedicine tool for ophthalmologic screening), and education throughout the unit.

Identification of an infant on admission was a major theme of improved hospitals. Some hospitals noted eligible infants on their admission notes and daily progress notes, some placed infants on a list in the unit at the time of birth, and some communicated with the ophthalmologist's office at the time of birth to be placed on a list at that office.

The use of Retcam or telemedicine was also common to the improved hospitals. These units identified difficulty with consistent ophthalmology exams prior to Retcam. Implementation of telemedicine and training of registered nurse (RN) and physician staff in its use made obtaining reliable exams much easier. Each unit now has a system whereby infants are examined once per week and the report of that infant is emailed back to the unit the following day. More frequent exams are also possible for infants with evolving ROP and email communication from the offsite ophthalmologist facilitates these extra exams.

Unit-wide education was an important component to the improved NICUs. These units found that when multiple layers of staff are familiar with the screening criteria as well as the disease process, fewer infants were missed. RN involvement was especially important as it served as a double check to physicians on rounds for infants who qualify.

Low Performing Hospitals

Themes found in the low performing hospital include difficulty identifying eligible neonates due to lack of education, older babies more commonly missed, and difficulty getting the ophthalmologist to come to the unit.

The low performing hospital identified education as an area that was needed for improvement. In this unit, the nurses do not often know the ROP screening criteria, and are thus not a part of the system to help identify neonates. Furthermore, some of the neonatologists were confused about the screening criteria itself and believed that infants qualified based on weight AND gestational age at birth, rather than weight OR gestational age at birth.

Likely due to lack of education, older babies were more often missed in the lower performing unit. These infants tended to be intrauterine growth restriction and qualified for ROP screening by weight, and not gestational age. Neonatologists admitted that they did not think of these infants as at risk due to their gestational age, although they did meet the screening criteria. The lower performing unit also noted difficulty with consistent access to an ophthalmologist.

Ophthalmologists' Perspective

As mentioned above, one common theme to successful NICU screening of ROP is having a committed ophthalmologist perform the exams routinely. However, all ophthalmologists we talked to identified several barriers to screening neonates for ROP. They suggested that many ophthalmologists are uncomfortable with the exam of preterm infants, and that fear of missing a case of ROP, or losing a patient to follow-up, could lead to subsequent liability. They also discussed how the length and set up of the exam is burdensome, especially in a busy practice where physicians are responsible for exams in multiple hospitals and clinics. Finally, a perception of poor reimbursement is a disincentive to offering this service to NICUs.

Despite these significant identified barriers, the ophthalmologists that we talked to were committed to providing their services to infants in the NICU. They discussed feeling like doing these exams was “a calling” or an obligation. They discussed their commitment to preserving the eyesight of premature infants. And they feel rewarded that their exams can truly impact a baby's quality of life. These motivators justify working past the barriers identified above. And the units that they work in help ease some burdens by ensuring that infants are set up for exams with paperwork and tools properly laid out, eyes dilated, and medications ready so that exams can be expedited. Narrative data from ophthalmologists on motivators, barriers, and facilitators to care are shown in [Table 1].

Table 1 Ophthalmologist facilitators, motivators, and barriers to care

Facilitators
Consistent communication	“My office receives a phone call from a contact person here and my technician calls me and says, “The County has two babies they need you to see this week. When can you do it?” And so I look at my schedule and say, “Well three days from now I'll be in that area. Just set it up.” And then I'll tell my technician, “I'll probably be able to be there by 12:00 so have them dilate”—usually 45 minutes before I get there… Then my staff puts it on my books”
Financial incentive	“Probably like anything else in this world, there'd probably have to be a financial incentive to get some doctors motivated to do that, especially if they're going to have to go spend some time with somebody, take time out of their schedule to brush up on their skills and spend possibly time and money away from their practice”
System/Procedures	“So we have a system in place that kids get dilated when I come, they're ready to go, dilated, the equipment is at the bedside, the nurses are helpful, the parents are educated. I really just come in, look at the eyes and write my report”
Motivators
A calling/Commitment	“I just feel a commitment to the kids. I really do. I went into pediatric ophthalmology thinking they need an advocate, the children… So I just feel like children need an advocate and if it's not me who'll do it, there's nobody else around here who will do it”
Moral obligation	“It's a moral responsibility that we have and when you've seen patients that were very premature and very small and, you know, all the odds against them and then you see them at three or four or five year and they're reading a chart, that is really—as pediatric ophthalmologists, it's the largest impact that you can see, from having a kid that was going to be blind—because we have now 10- and 20-year-olds that are now blind from ROP—to a kid that could have been blinded and now constantly is doing very well”
Rewarding	“So it's one of the things that, as pediatric ophthalmologists, we see the biggest impact, the biggest change in quality of life, and it's very rewarding”
Barriers
Fear	“There is fear for people that if you get involved, you have a lot of liability over time because there's been cases that are very public, not that many but they get a lot of attention, so that makes people fearful of getting involved even when they're training to prepare them to do this”
Discomfort	“Visualization in the infant eye is difficult sometimes, like looking through a watery glass and you don't really get a super clear view like you would expect. So I think a lot of doctors who don't do a lot of them do feel uncomfortable with it and feel like you're missing something”
Liability	“In fact several years ago, there was a several multimillion dollar settlement that scared a lot of the ophthalmologists… So at that time I was advised to ask the hospitals to see if they would indemnify me to do it and they don't and they haven't and we're still doing it”
Time	“There are some days—I go to 12 locations and each half day is a different location and each week on the calendar looks different than the week before or after. So I'll usually find a little hole somewhere between surgery in the morning, office in the afternoon and if I don't, then I'll come in extremely early or extremely late… That's how I've been doing it for 20 years”

Discussion

This study highlights differences in NICUs that consistently screen all eligible neonates for ROP and those that have lower screening rates. Successful screening of neonates is twofold. First, eligible infants must be identified for screening. Second, screening must occur at the proper time. Synthesis of key factors for ROP screening is shown in [Fig. 1]. This study identified barriers and successes in each of these steps.

To identify eligible neonates, NICU nurses and physicians must know the screening criteria. We found that the hospital identified as a low performer missed an infant who qualified based on weight but not gestational age. Similarly, a recent survey of NICU directors showed that there is ongoing confusion about the ROP screening criteria. In this survey, 97% of respondents report using the gestational age as screening criterion. However, only 80% reported using the birth weight as a criterion.[6] Unit-wide education is essential in supporting the multidisciplinary system of double checks required to ensure proper identification of eligible neonates. The improved hospitals in this study also demonstrated that identification of eligible neonates on admission helps ensure that they are not missed once their eye exams are due.

Once identified, barriers still exist that prevent proper screening for ROP. One major barrier identified is inadequate access to an ophthalmologist. Previous studies have supported this finding that there is a dearth of ophthalmologists willing to screen for ROP.[6] [7] Our discussions with ophthalmologists punctuate the challenges faced by those in this profession in making ROP screening a part of their career. Like in this study, a 2006 American Academy of Ophthalmology survey also showed medical liability, reimbursement, and lack of hospital support as barriers.[6] As described by interviews with ophthalmologists in this study, bedside preparation to limit ophthalmologist's time required in the NICU helps address one concern. Improved malpractice provided by hospitals and improved reimbursement are other potential solutions.

Telemedicine is a new innovation that has been identified as an adjunct to in-person ROP screening.[8] Studies have shown telemedicine for ROP to be both safe and effective.[9] [10] It may help address ongoing unmet clinical needs seen in ROP screening. Furthermore, due to the subjective nature of ROP screening, having one offsite ophthalmologist might also decrease variation in interpretations of retinal images.[11]

This study has several limitations. We were only able to thoroughly interview teams from six NICUs in California. While this small number of NICUs may not be representative of all NICUs in California, there are still important lessons learned from each interview. Furthermore, because low performing hospitals were less likely to agree to interviews for the study, we only had one site visit for a low performing hospital. However, lessons from the improved hospitals such as identification of eligible neonates on admission and the use of Retcam could address the problems found in the low performing hospital. This leads us to believe that there is some validity to our findings in that one unit. Future research may include a quantitative survey on ROP screening practices of all units in California to obtain a higher level overview.

Success in ROP screening is multifactorial. NICUs that struggle to consistently identify and screen neonates for ROP could consider unit-wide education of the screening criteria, implementing systems with double checks and reminders, and use of a multidisciplinary team with overlap of accountability. Units with difficulty accessing ophthalmologists might benefit from telemedicine such as Retcam. With increased numbers of surviving premature infants, our need to identify and screen neonates will continue to rise. This study offers some strategies to ensure successful screening for ROP moving forward.

Conflict of Interest

None.

Acknowledgment

We appreciate the Medical Directors and staff who were willing to be interviewed in participating NICUs.

• References

• 1 Yonekawa Y, Thomas BJ, Thanos A. , et al. The cutting edge of retinopathy of prematurity care: expanding the boundaries of diagnosis and treatment. Retina 2017; 37 (12) 2208-2225
• 2 Bain LC, Dudley RA, Gould JB, Lee HC. Factors associated with failure to screen newborns for retinopathy of prematurity. J Pediatr 2012; 161 (05) 819-823
• 3 Ludwig CA, Chen TA, Hernandez-Boussard T, Moshfeghi AA, Moshfeghi DM. The epidemiology of retinopathy of prematurity in the United States. Ophthalmic Surg Lasers Imaging Retina 2017; 48 (07) 553-562
• 4 Foley G, Timonen V. Using grounded theory method to capture and analyze health care experiences. Health Serv Res 2015; 50 (04) 1195-1210
• 5 Glaser BG. The constant comparative method of qualitative analysis. Social Problems 1965; 12 (04) 436-445
• 6 Vartanian RJ, Besirli CG, Barks JD, Andrews CA, Musch DC. Trends in the screening and treatment of retinopathy of prematurity. Pediatrics 2017; 139 (01) e20161978
• 7 Kemper AR, Wallace DK, Quinn GE. Systematic review of digital imaging screening strategies for retinopathy of prematurity. Pediatrics 2008; 122 (04) 825-830
• 8 Weaver DT, Murdock TJ. Telemedicine detection of type 1 ROP in a distant neonatal intensive care unit. J AAPOS 2012; 16 (03) 229-233
• 9 Wang SK, Callaway NF, Wallenstein MB, Henderson MT, Leng T, Moshfeghi DM. SUNDROP: six years of screening for retinopathy of prematurity with telemedicine. Can J Ophthalmol 2015; 50 (02) 101-106
• 10 Quinn GE, Ying GS, Daniel E. , et al; e-ROP Cooperative Group. Validity of a telemedicine system for the evaluation of acute-phase retinopathy of prematurity. JAMA Ophthalmol 2014; 132 (10) 1178-1184
• 11 Gschließer A, Stifter E, Neumayer T. , et al. Inter-expert and intra-expert agreement on the diagnosis and treatment of retinopathy of prematurity. Am J Ophthalmol 2015; 160 (03) 553-560.e3