Pediatric Predictive Artificial Intelligence Implemented in Clinical Practice from 2010 to 2021: A Systematic Review

Abstract

Objective

To review pediatric artificial intelligence (AI) implementation studies from 2010 to 2021 and analyze reported performance measures.

Methods

We searched PubMed/Medline, Embase CINHAL, Cochrane Library CENTRAL, IEEE, and Web of Science with controlled vocabulary. Inclusion criteria: AI intervention in a pediatric clinical setting that learns from data (i.e., data-driven, as opposed to rule-based) and takes actions to make patient-specific recommendations; published between 01/2010 and 10/2021; must have agency (AI must provide guidance that affects clinical care, not merely running in the background). We extracted study characteristics, target users, implementation setting, time span, and performance measures.

Results

Of 126 articles reviewed as full text, 17 met inclusion criteria. Eight studies (47%) reported both clinical outcomes and process measures, six (35%) reported only process measures and two (12%) reported only clinical outcomes. Five studies (30%) reported no difference in clinical outcomes with AI, four (24%) reported improvement in clinical outcomes compared with controls, two (12%) reported positive effects on clinical outcomes with use of AI but had no formal comparison or controls, and one (6%) reported poor clinical outcomes with AI. Twelve studies (71%) reported improvement in process measures, while two (12%) reported no improvement. Five (30%) studies reported on at least 1 human performance measure.

Conclusion

While there are many published pediatric AI models, the number of AI implementations is minimal with no standardized reporting of outcomes, care processes, or human performance measures. More comprehensive evaluations will help elucidate mechanisms of impact.

Keywords

artificial intelligence - pediatrics - evaluation

• References

• 1 Hoodbhoy Z, Masroor Jeelani S, Aziz A. et al. Machine learning for child and adolescent health: a systematic review. Pediatrics 2021; 147 (01) e2020011833
• 2 Kwok TC, Henry C, Saffaran S. et al. Application and potential of artificial intelligence in neonatal medicine. Semin Fetal Neonatal Med 2022; 27 (05) 101346
• 3 Sharma M, Savage C, Nair M, Larsson I, Svedberg P, Nygren JM. Artificial intelligence applications in health care practice: scoping review. J Med Internet Res 2022; 24 (10) e40238
• 4 Iancu A, Leb I, Prokosch HU, Rödle W. Machine learning in medication prescription: a systematic review. Int J Med Inform 2023; 180: 105241
• 5 Alsabri M, Aderinto N, Mourid MR. et al. Artificial intelligence for pediatric emergency medicine. J Med Surg Public Health 2024; 3: 100137
• 6 Walczak S, Scorpio RJ. Predicting pediatric length of stay and acuity of care in the first ten minutes with artificial neural networks. Pediatr Crit Care Med 2000; 1 (01) 42-47
• 7 Hogan AH, Brimacombe M, Mosha M, Flores G. Comparing artificial intelligence and traditional methods to identify factors associated with pediatric asthma readmission. Acad Pediatr 2022; 22 (01) 55-61
• 8 Carlin CS, Ho LV, Ledbetter DR, Aczon MD, Wetzel RC. Predicting individual physiologically acceptable states at discharge from a pediatric intensive care unit. J Am Med Inform Assoc 2018; 25 (12) 1600-1607
• 9 Aczon MD, Ledbetter DR, Laksana E, Ho LV, Wetzel RC. Continuous prediction of mortality in the PICU: a recurrent neural network model in a single-center dataset. Pediatr Crit Care Med 2021; 22 (06) 519-529
• 10 Lamping F, Jack T, Rübsamen N. et al. Development and validation of a diagnostic model for early differentiation of sepsis and non-infectious SIRS in critically ill children - a data-driven approach using machine-learning algorithms. BMC Pediatr 2018; 18 (01) 112
• 11 Le S, Hoffman J, Barton C. et al. Pediatric severe sepsis prediction using machine learning. Front Pediatr 2019; 7: 413
• 12 Mayampurath A, Sanchez-Pinto LN, Hegermiller E. et al. Development and external validation of a machine learning model for prediction of potential transfer to the PICU. Pediatr Crit Care Med 2022; 23 (07) 514-523
• 13 Pollack MM, Holubkov R, Berg RA. et al. Eunice Kennedy Shriver National Institute of Child Health and Human Development Collaborative Pediatric Critical Care Research Network (CPCCRN). Predicting cardiac arrests in pediatric intensive care units. Resuscitation 2018; 133: 25-32
• 14 McInnis C, Garcia MJS, Widjaja E. et al. Magnetic resonance imaging findings are associated with long-term global neurological function or death after traumatic brain injury in critically ill children. J Neurotrauma 2021; 38 (17) 2407-2418
• 15 Tunthanathip T, Oearsakul T. Application of machine learning to predict the outcome of pediatric traumatic brain injury. Chin J Traumatol 2021; 24 (06) 350-355
• 16 Goto Y, Maeda T, Nakatsu-Goto Y. Decision tree model for predicting long-term outcomes in children with out-of-hospital cardiac arrest: a nationwide, population-based observational study. Crit Care 2014; 18 (03) R133
• 17 Fung FW, Topjian AA, Xiao R, Abend NS. Early EEG features for outcome prediction after cardiac arrest in children. J Clin Neurophysiol 2019; 36 (05) 349-357
• 18 Winter MC, Day TE, Ledbetter DR. et al. Machine learning to predict cardiac death within 1 hour after terminal extubation. Pediatr Crit Care Med 2021; 22 (02) 161-171
• 19 Sujan M, Pool R, Salmon P. Eight human factors and ergonomics principles for healthcare artificial intelligence. BMJ Health Care Inform 2022; 29 (01) 100516
• 20 Page MJ, McKenzie JE, Bossuyt PM. et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021; 372 (71) n71
• 21 Niel O, Bastard P, Boussard C, Hogan J, Kwon T, Deschênes G. Artificial intelligence outperforms experienced nephrologists to assess dry weight in pediatric patients on chronic hemodialysis. Pediatr Nephrol 2018; 33 (10) 1799-1803
• 22 Amrit C, Paauw T, Aly R, Lavric M. Identifying child abuse through text mining and machine learning. Expert systems with applications. Expert Syst Appl 2017; 88: 402-418
• 23 Achten NB, Dorigo-Zetsma JW, van der Linden PD, van Brakel M, Plötz FB. Sepsis calculator implementation reduces empiric antibiotics for suspected early-onset sepsis. Eur J Pediatr 2018; 177 (05) 741-746
• 24 van de Maat J, van der Ven M, Driessen G. et al. Cost study of a cluster randomized trial on a clinical decision rule guiding antibiotic treatment in children with suspected lower respiratory tract infections in the emergency department. Pediatr Infect Dis J 2020; 39 (11) 1026-1031
• 25 Stephens TN, Joerin A, Rauws M, Werk LN. Feasibility of pediatric obesity and prediabetes treatment support through Tess, the AI behavioral coaching chatbot. Transl Behav Med 2019; 9 (03) 440-447
• 26 Clark MM, Hildreth A, Batalov S. et al. Diagnosis of genetic diseases in seriously ill children by rapid whole-genome sequencing and automated phenotyping and interpretation. Sci Transl Med 2019; 11 (489) eaat6177
• 27 Smith JC, Spann A, McCoy AB. et al. Natural language processing and machine learning to enable clinical decision support for treatment of pediatric pneumonia. AMIA Annu Symp Proc 2021; 2020: 1130-1139
• 28 De La Vega FM, Chowdhury S, Moore B. et al. Artificial intelligence enables comprehensive genome interpretation and nomination of candidate diagnoses for rare genetic diseases. Genome Med 2021; 13 (01) 153
• 29 van de Maat JS, Peeters D, Nieboer D. et al. Evaluation of a clinical decision rule to guide antibiotic prescription in children with suspected lower respiratory tract infection in The Netherlands: a stepped-wedge cluster randomised trial. PLoS Med 2020; 17 (01) e1003034
• 30 Frymoyer A, Schwenk HT, Zorn Y. et al. Model-informed precision dosing of vancomycin in hospitalized children: implementation and adoption at an academic children's hospital. Front Pharmacol 2020; 11: 551
• 31 Ni Y, Bermudez M, Kennebeck S, Liddy-Hicks S, Dexheimer J. A real-time automated patient screening system for clinical trials eligibility in an emergency department: design and evaluation. JMIR Med Inform 2019; 7 (03) e14185
• 32 Li X, Tian D, Li W. et al. Artificial intelligence-assisted reduction in patients' waiting time for outpatient process: a retrospective cohort study. BMC Health Serv Res 2021; 21 (01) 237
• 33 Dexheimer JW, Abramo TJ, Arnold DH. et al. Implementation and evaluation of an integrated computerized asthma management system in a pediatric emergency department: a randomized clinical trial. Int J Med Inform 2014; 83 (11) 805-813
• 34 Seol HY, Shrestha P, Muth JF. et al. Artificial intelligence-assisted clinical decision support for childhood asthma management: A randomized clinical trial. PLoS One 2021; 16 (08) e0255261
• 35 de Vos-Kerkhof E, Nijman RG, Vergouwe Y. et al. Impact of a clinical decision model for febrile children at risk for serious bacterial infections at the emergency department: a randomized controlled trial. PLoS One 2015; 10 (05) e0127620
• 36 Lin H, Li R, Liu Z. et al. Diagnostic efficacy and therapeutic decision-making capacity of an artificial intelligence platform for childhood cataracts in eye clinics: a multicentre randomized controlled trial. EClinicalMedicine 2019; 9: 52-59
• 37 Nimri R, Battelino T, Laffel LM. et al. NextDREAM Consortium. Insulin dose optimization using an automated artificial intelligence-based decision support system in youths with type 1 diabetes. Nat Med 2020; 26 (09) 1380-1384
• 38 Longhurst CA, Singh K, Chopra A, Atreja A, Brownstein JS. A call for artificial intelligence implementation science centers to evaluate clinical effectiveness. NEJM AI 2024; 1 (08) AIp2400223
• 39 Bright TJ, Wong A, Dhurjati R. et al. Effect of clinical decision-support systems: a systematic review. Ann Intern Med 2012; 157 (01) 29-43
• 40 Dziorny AC, Heneghan JA, Bhat MA. et al. Pediatric Data Science and Analytics (PEDAL) Subgroup of the Pediatric Acute Lung Injury and Sepsis Investigators (PALISI) Network. Clinical decision support in the PICU: implications for design and evaluation. Pediatr Crit Care Med 2022; 23 (08) e392-e396
• 41 Lundberg S, Lee SI. A Unified Approach to Interpreting Model Predictions. Advances in Neural Information Processing Systems [Internet]. pp. 4766–4775. Accessed 2017 at: http://arxiv.org/abs/1705.07874
• 42 Collins GS, Moons KGM. Reporting of artificial intelligence prediction models. Lancet 2019; 393 (10181): 1577-1579
• 43 Sounderajah V, Ashrafian H, Aggarwal R. et al. Developing specific reporting guidelines for diagnostic accuracy studies assessing AI interventions: the STARD-AI steering group. Nat Med 2020; 26 (06) 807-808
• 44 Vasey B, Nagendran M, Campbell B. et al. Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. Nature Medicine 2022 28:5 [Internet];28(5):924–33. Accessed May 18, 2022 at: https://www.nature.com/articles/s41591-022-01772-9
• 45 Liu X, Rivera SC, Moher D, Calvert MJ, Denniston AK. SPIRIT-AI and CONSORT-AI Working Group. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. BMJ 2020; 370: m3164
• 46 Muralidharan V, Burgart A, Daneshjou R, Rose S. Recommendations for the use of pediatric data in artificial intelligence and machine learning ACCEPT-AI. npj. Digit Med 2023; 6 (01) 1-6
• 47 Cabitza F, Campagner A. The need to separate the wheat from the chaff in medical informatics: Introducing a comprehensive checklist for the (self)-assessment of medical AI studies. Int J Med Inform 2021; 153: 104510
• 48 Tu C, Russo A, Zhang Y. A Checklist for the Usability Evaluation of Artificial Intelligence (AI) mHealth Applications Graphical User Interface. In: Design, User Experience, and Usability: 13th International Conference, DUXU 2024, Held as Part of the 26th HCI International Conference, HCII 2024, Washington, DC, USA. June 29–July 4, 2024, Proceedings, Part I [Internet]. Berlin, Heidelberg: Springer-Verlag. p. 324–337. Accessed November 4, 2024 at: https://doi.org/10.1007/978-3-031-61351-7_23