PDF icon Download PDF
Am J Perinatol 2022; 39(02): 195-203
DOI: 10.1055/s-0040-1715530
Original Article

Validation of an Instrument for Real-Time Assessment of Neonatal Intubation Skills: A Randomized Controlled Simulation Study

Lindie J.M.K. Kuijpers‡   *
1   Department of Neonatology, Radboud University Medical Center Amalia Children's Hospital, Nijmegen, The Netherlands
,
Mathijs Binkhorst  ‡   *
1   Department of Neonatology, Radboud University Medical Center Amalia Children's Hospital, Nijmegen, The Netherlands
,
Nicole K. Yamada
2   Division of Neonatal and Developmental Medicine, Department of Pediatrics, Center for Advanced Pediatric and Perinatal Education (CAPE), Stanford University School of Medicine, Palo Alto, California
,
Romy N. Bouwmeester
1   Department of Neonatology, Radboud University Medical Center Amalia Children's Hospital, Nijmegen, The Netherlands
,
Arno F.J. van Heijst
1   Department of Neonatology, Radboud University Medical Center Amalia Children's Hospital, Nijmegen, The Netherlands
,
Louis P. Halamek
2   Division of Neonatal and Developmental Medicine, Department of Pediatrics, Center for Advanced Pediatric and Perinatal Education (CAPE), Stanford University School of Medicine, Palo Alto, California
,
Marije Hogeveen
1   Department of Neonatology, Radboud University Medical Center Amalia Children's Hospital, Nijmegen, The Netherlands
› Author Affiliations
Funding The study was partially supported by the Endowment for the Center for Advanced Pediatric and Perinatal Education (CAPE). The project was also supported by the Agency for Healthcare Research and Quality (grant number P30HS023506). The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality.
› Further Information
Also available ateRefHebammen
   Permissions and Reprints
Preview

Abstract

Objective This study aimed to evaluate the construct validity and reliability of real-time assessment of a previously developed neonatal intubation scoring instrument (NISI).

Study Design We performed a randomized controlled simulation study at a simulation-based research and training facility. Twenty-four clinicians experienced in neonatal intubation (“experts”) and 11 medical students (“novices”) performed two identical elective intubations on a neonatal patient simulator. Subjects were randomly assigned to either the intervention group, receiving predefined feedback between the two intubations, or the control group, receiving no feedback. Using the previously developed NISI, all intubations were assessed, both in real time and remotely on video. Construct validity was evaluated by (1) comparing the intubation performances, expressed as percentage scores, with and without feedback, and (2) correlating the intubation performances with the subjects' level of experience. The intrarater reliability, expressed as intraclass correlation coefficient (ICC), of real-time assessment compared with video-based assessment was determined.

Results The intervention group contained 18 subjects, the control group 17. Background characteristics and baseline intubation scores were comparable in both groups. The median (IQR) change in percentage scores between the first and second intubation was significantly different between the intervention and control group (11.6% [4.7–22.8%] vs. 1.4% [0.0–5.7%], respectively; p = 0.013). The 95% CI for this 10.2% difference was 2.2 to 21.4%. The subjects' experience level correlated significantly with their percentage scores (Spearman's R = 0.70; p <0.01). ICC's were 0.95 (95% CI: 0.89–0.97) and 0.94 (95% CI: 0.89–0.97) for the first and second intubation, respectively.

Conclusion Our NISI has construct validity and is reliable for real-time assessment.

Key Points

  • Our neonatal intubation scoring instrument has construct validity.

  • Our instrument can be reliably employed to assess neonatal intubation skills directly in real time.

  • It is suitable for formative assessment, i.e., providing direct feedback during procedural training.

Keywords

education - intubation - neonatal - simulation-based training - scoring instrument

Note

This study was approved by the Institutional Review Board Committee at Stanford University (#44321). All subjects provided informed written consent. The ISRCTN registry trial ID is ISRCTN47516971.


Authors' Contributions

Data collection was done by L.J.M.K.K., N.K.Y., and L.P.H. Data analysis and interpretation was done by L.J.M.K.K., M.B., and M.H. Initial manuscript was drafted by L.J.M.K.K., and M.B. The manuscript was critically reviewed and finally approved by all the authors. All authors agree to be accountable for all the aspects of the work, had full access to all of the data, and can take responsibility for the integrity and accuracy of data analysis and reporting. Study conceptualization and design was done by all the authors.


* These authors contributed equally to this work.


Supplementary Material



Publication History

Received: 07 May 2020

Accepted: 07 July 2020

Article published online:
08 September 2020

© 2020. Thieme. All rights reserved.

Thieme Medical Publishers, Inc.
333 Seventh Avenue, 18th Floor, New York, NY 10001, USA