External Validation of a Clinical Nomogram for Predicting Intracranial Hematoma Following Head Computed Tomography in Pediatric Traumatic Brain Injury

Abstract

Introduction Over-investigation of head computed tomography (CT) has been observed in children with TBI. Long-term effects from a head CT brain scan have been addressed and those should be balanced. A nomogram is a simple prediction tool that has been reported for predicting intracranial injuries following a head CT of the brain in TBI children in literature. This study aims to validate the performance of the nomogram using unseen data. Additionally, the secondary objective aims to estimate the net benefit of the nomogram by decision curve analysis (DCA).

Methods We conducted a retrospective cohort study with 64 children who suffered from traumatic brain injury (TBI) and underwent a CT of the brain. Nomogram's scores were assigned according to various variables in each patient; therefore sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), accuracy and F1 score were estimated by the cross-tabulation of the actual results and the predicted results. Additionally, the benefits of a nomogram were compared with “None” and “All” protocols using DCA.

Results There were 64 children with TBI who underwent a head CT in the present study. From the cross-tabulation, the nomogram had a sensitivity of 0.60 (95%CI 0.29–0.90), specificity of 0.96 (0.91–1.0), PPV of 0.75 (0.44–1.0), NPV of 0.92 (0.86–0.99), accuracy of 0.90 (0.83–0.97), and an F1 score of 0.66 (0.59–0.73). Also, the area under the curve was 0.78 which was defined as acceptable performance. For the DCA at 0.1 high-risk threshold, the net benefit of the nomogram was 0.75, whereas the “All” protocol had the net benefit of 0.40 which was obviously different.

Conclusion A nomogram is a suitable method as an alternative prediction tool in general practice that has advantages over other protocols.

Resumo

Introdução A investigação excessiva da tomografia computadorizada (TC) de crânio tem sido observada em crianças com TCE. Os efeitos a longo prazo de uma tomografia computadorizada de crânio foram abordados e devem ser equilibrados. Um nomograma é uma ferramenta de predição simples que foi relatada na literatura para prever lesões intracranianas após uma tomografia computadorizada de crânio em crianças com TCE. Este estudo tem como objetivo validar o desempenho do nomograma usando dados não vistos. Adicionalmente, o objetivo secundário visa estimar o benefício líquido do nomograma por meio da análise da curva de decisão (DCA).

Métodos Realizamos um estudo de coorte retrospectivo com 64 crianças que sofreram traumatismo cranioencefálico (TCE) e foram submetidas a tomografia computadorizada de crânio. As pontuações do Nomograma foram atribuídas de acordo com diversas variáveis em cada paciente; portanto, sensibilidade, especificidade, valor preditivo positivo (VPP), valor preditivo negativo (VPN), acurácia e escore F1 foram estimados pela tabulação cruzada dos resultados reais e dos resultados previstos. Além disso, os benefícios de um nomograma foram comparados com os protocolos “Nenhum” e “Todos” usando DCA.

Resultados Houve 64 crianças com TCE que foram submetidas a tomografia computadorizada de crânio no presente estudo. A partir da tabulação cruzada, o nomograma apresentou sensibilidade de 0,60 (IC95% 0,29–0,90), especificidade de 0,96 (0,91–1,0), VPP de 0,75 (0,44–1,0), VPN de 0,92 (0,86–0,99), acurácia de 0,90 (0,83–0,97) e uma pontuação F1 de 0,66 (0,59–0,73). Além disso, a área sob a curva foi de 0,78, definida como desempenho aceitável. Para o DCA no limiar de alto risco de 0,1, o benefício líquido do nomograma foi de 0,75, enquanto o protocolo “Todos” teve o benefício líquido de 0,40, o que foi obviamente diferente.

Conclusão Um nomograma é um método adequado como ferramenta alternativa de predição na prática geral que apresenta vantagens sobre outros protocolos.

Keywords

External validation - Nomogram - intracranial hematoma - pediatric traumatic brain injury - head injury

Palavras-chave

validação externa - nomograma - hematoma intracraniano - lesão cerebral traumática pediátrica - ferimento na cabeça

Abbreviations Used in this Paper

AUC: Area under the curve, CT: computed tomography, DCA: decision curve analysis, GCS: Glasgow Coma Scale, GOS: Glasgow Outcome Scale, IQR: interquartile range, NPV: negative predictive value, PPV: positive predictive value, ROC: Receiver operating characteristic, SD: standard deviation, SPIN: Specific test when Positive rules IN the disease, TBI: traumatic brain injury

Declarations

All procedures performed in the study that involved studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee or both and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Author Contributions

AJ and TT conceived the study and designed the method. TT supervised the conduct of the data collection. AJ and TT managed the data, including quality control. AJ and TT provided statistical advice on the study design and analyzed the data and AJ drafted the manuscript, and TT contributed substantially to its revision. TT takes responsibility for the paper.

Publikationsverlauf

Eingereicht: 29. März 2021

Angenommen: 16. Juni 2021

Artikel online veröffentlicht:
29. September 2023

• References

• 1 The Centers for Disease Control and Prevention. CDC newsroom [internet]. 2020 [cited 2020 Jul 5]. Available from: https://www.cdc.gov/media/releases/2018/p0904-tbi-guidelines.html
• 2 Taweesomboonyat T, Kaewborisutsakul A, Tunthanathip T, Saeheng S, Oearsakul T. Necessity of In-hospital Neurological Observation for Mild Traumatic Brain Injury Patients with Negative Computed Tomography Brain Scans. J Health Sci Med Res 2000; 38: 267-274
• 3 Larson DB, Johnson LW, Schnell BM, Salisbury SR, Forman HP. National trends in CT use in the emergency department: 1995-2007. Radiology 2011; 258 (01) 164-173
• 4 Sheppard JP, Nguyen T, Alkhalid Y, Beckett JS, Salamon N, Yang I. Risk of brain tumour induction from pediatric head CT procedures: A systematic literature review. Brain Tumor Res Treat 2018; 6 (01) 1-7
• 5 Pearce MS, Salotti JA, Little MP. et al. Radiation exposure from CT scans in childhood and subsequent risk of leukaemia and brain tumours: a retrospective cohort study. Lancet 2012; 380 (9840): 499-505
• 6 Dunning J, Daly JP, Lomas JP, Lecky F, Batchelor J, Mackway-Jones K. Children's head injury algorithm for the prediction of important clinical events study group. Derivation of the children's head injury algorithm for the prediction of important clinical events decision rule for head injury in children. Arch Dis Child 2006; 91 (11) 885-891
• 7 Osmond MH, Klassen TP, Wells GA. et al; Pediatric Emergency Research Canada (PERC) Head Injury Study Group. CATCH: a clinical decision rule for the use of computed tomography in children with minor head injury. CMAJ 2010; 182 (04) 341-348
• 8 Kuppermann N, Holmes JF, Dayan PS. et al; Pediatric Emergency Care Applied Research Network (PECARN). Identification of children at very low risk of clinically-important brain injuries after head trauma: a prospective cohort study. Lancet 2009; 374 (9696): 1160-1170
• 9 Tunthanathip T, Duangsuwan J, Wattanakitrungroj N, Tongman S, Phuenpathom N. Clinical nomogram predicting intracranial injury in pediatric traumatic brain injury. J Pediatr Neurosci 2020; 15 (04) 409-415
• 10 Tunthanathip T, Ratanalert S, Sae-Heng S. et al. Prognostic Factors and Nomogram Predicting Survival in Diffuse Astrocytoma. J Neurosci Rural Pract 2020; 11 (01) 135-143
• 11 Tunthanathip T, Phuenpathom N. Impact of road traffic injury to pediatric traumatic brain injury in southern Thailand. J Neurosci Rural Pract 2017; 8 (04) 601-608
• 12 Marques A, Almeida S, Carvalho J, Cruz J, Oliveira A, Jácome C. Reliability, Validity, and Ability to Identify Fall Status of the Balance Evaluation Systems Test, Mini-Balance Evaluation Systems Test, and Brief-Balance Evaluation Systems Test in Older People Living in the Community. Arch Phys Med Rehabil 2016; 97 (12) 2166-2173.e1
• 13 Tunthanathip T, Oearsakul T. Machine Learning Approaches for Prognostication of Newly Diagnosed Glioblastoma. Int J Nutr Pharmacol Neurol Dis 2021; 11: 57-63
• 14 Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making 2006; 26 (06) 565-574 Doi: 10.1177/0272989. Doi: × 06295361
• 15 Kerr KF, Brown MD, Zhu K, Janes H. Assessing the Clinical Impact of Risk Prediction Models With Decision Curves: Guidance for Correct Interpretation and Appropriate Use. J Clin Oncol 2016; 34 (21) 2534-2540
• 16 Van Calster B, Wynants L, Verbeek JFM. et al. Reporting and Interpreting Decision Curve Analysis: A Guide for Investigators. Eur Urol 2018; 74 (06) 796-804
• 17 Austin PC, van Klaveren D, Vergouwe Y, Nieboer D, Lee DS, Steyerberg EW. Validation of prediction models: examining temporal and geographic stability of baseline risk and estimated covariate effects. Diagn Progn Res 2017; 1: 12
• 18 Girardat-Rotar L, Braun J, Puhan MA, Abraham AG, Serra AL. Temporal and geographical external validation study and extension of the Mayo Clinic prediction model to predict eGFR in the younger population of Swiss ADPKD patients. BMC Nephrol 2017; 18 (01) 241
• 19 Baeyens JP, Serrien B, Goossens M, Clijsen R. Questioning the “SPIN and SNOUT” rule in clinical testing. Arch Physiother 2019; 9: 4
• 20 Tunthanathip T, Udomwitthayaphiban S. Development and Validation of a Nomogram for Predicting the Mortality after Penetrating Traumatic Brain Injury. Bull Emerg Trauma 2019; 7 (04) 347-354
• 21 Devin CJ, Bydon M, Alvi MA. et al. A predictive model and nomogram for predicting return to work at 3 months after cervical spine surgery: an analysis from the Quality Outcomes Database. Neurosurg Focus 2018; 45 (05) E9
• 22 Tunthanathip T, Sae-Heng S, Oearsakul T, Sakarunchai I, Kaewborisutsakul A, Taweesomboonyat C. Machine learning applications for the prediction of surgical site infection in neurological operations. Neurosurg Focus 2019; 47 (02) E7
• 23 Wood T. What is the F-score? [Internet]. 2021 [cited 2021 Mar 5]. Available from: https://deepai.org/machine-learning-glossary-and-terms/f-score
• 24 Shung KP. Accuracy, Precision, Recall or F1? [Internet]. 2018 [cited 2021 Mar 5]. Available from: https://towardsdatascience.com/accuracy-precision-recall-or-f1-331fb37c5cb9
• 25 Toll DB, Janssen KJ, Vergouwe Y, Moons KG. Validation, updating and impact of clinical prediction rules: a review. J Clin Epidemiol 2008; 61 (11) 1085-1094
• 26 Moons KG, Kengne AP, Grobbee DE. et al. Risk prediction models: II. External validation, model updating, and impact assessment. Heart 2012; 98 (09) 691-698