Incorporating AI-Driven Vision Systems to Quantify Learning Curve in EVD Placement

Abstract

Objectives

There is an application for artificial intelligence (AI) to augment medical education. The aim of this study was to incorporate AI-powered cameras to quantify the learning curve and performance metrics associated with external ventricular drain (EVD) placement.

Methods

Fourteen participants, comprising medical students and neurosurgical residents, were recorded performing an EVD on a trainer head. Five panoramic cameras were installed within the simulation suite. The model employed convolutional neural networks to track anatomical landmarks and assess task completion. Quantification of the learning curve was achieved by aggregating scores across three phases: preparation, insertion, and closing. Additional metrics included fluidity, a proxy for surgical finesse.

Results

The model successfully itemized parameters that characterize EVD placement. The study demonstrated a clear learning curve in EVD placement. The overall scores were 64.4/126 (51.1%), 99.6/126 (79%), and 113/126 (89.7%) for the students, junior residents, and senior residents (p < 0.0001). Significant improvements were observed in the preparation, insertion, and closing phases. The mean scores for preparation were 16.4/37 (44.3%), 25.6/37 (69.2), and 30.5/37 (82.4) for the students, junior residents, and senior residents (p < 0.0001). The mean scores for insertion were 26.2/44 (59.5%), 37.8/44 (85.9%), and 38.5/44 (87.5%) for the students, junior residents, and senior residents (p = 0.026). The mean scores during closing were 13/25 (52%), 22.2/25 (88.8%), and 25/25 (100%) for the students, junior residents, and senior residents (p = 0.0034). Fluidity improved significantly with training level (p = 0.0006).

Conclusion

Our platform effectively quantified the learning curve associated with EVD placement, underscoring the importance of objective feedback and AI's potential to facilitate skill acquisition.

Keywords

artificial intelligence - vision systems - neurosurgery - skull base - EVD - machine learning

Contributors'

Conception and design: R.D.S., A.M., N.M., K.H., E.S., and J.J.E. Acquisition of data and reviewing submitted version of manuscript: R.D.S., A.M., N.M., K.H., E.S., D.P.B., and J.J.E. Analysis and interpretation of data: R.D.S. and E.S. Drafting the article: R.D.S., A.M., and J.J.E. Critically revising the article: R.D.S., A.M., N.M., K.H., D.P.B., and J.J.E. Statistical analysis: R.D.S., A.M., and E.S.

Publication History

Received: 04 April 2025

Accepted: 23 June 2025

Accepted Manuscript online:
24 June 2025

Article published online:
11 July 2025

• References

• 1 Siler DA, Cleary DR, Tonsfeldt KJ. et al. Physiological responses and training satisfaction during national rollout of a neurosurgical intraoperative catastrophe simulator for resident training. Oper Neurosurg (Hagerstown) 2023; 24 (01) 80-87
• 2 Bina RW, Lemole Jr GM, Dumont TM. On resident duty hour restrictions and neurosurgical training: review of the literature. J Neurosurg 2016; 124 (03) 842-848
• 3 Burkhardt E, Adeeb N, Terrell D. et al. Factors impacting neurosurgery residents' operative case volume: a nationwide survey. J Neurosurg 2023; 140 (02) 570-575
• 4 Dumont TM, Rughani AI, Penar PL, Horgan MA, Tranmer BI, Jewell RP. Increased rate of complications on a neurological surgery service after implementation of the Accreditation Council for Graduate Medical Education work-hour restriction. J Neurosurg 2012; 116 (03) 483-486
• 5 Bissonnette V, Mirchi N, Ledwos N, Alsidieri G, Winkler-Schwartz A, Del Maestro RF. Neurosurgical Simulation & Artificial Intelligence Learning Centre. Artificial intelligence distinguishes surgical training levels in a virtual reality spinal task. J Bone Joint Surg Am 2019; 101 (23) e127
• 6 Haug CJ, Drazen JM. Artificial intelligence and machine learning in clinical medicine, 2023. N Engl J Med 2023; 388 (13) 1201-1208
• 7 Kiyasseh D, Ma R, Haque TF. et al. A vision transformer for decoding surgeon activity from surgical videos. Nat Biomed Eng 2023; 7 (06) 780-796
• 8 Russ S, Arora S, Wharton R. et al. Measuring safety and efficiency in the operating room: development and validation of a metric for evaluating task execution in the operating room. J Am Coll Surg 2013; 216 (03) 472-481
• 9 Gonzalez-Romo NI, Hanalioglu S, Mignucci-Jiménez G. et al. Quantification of motion during microvascular anastomosis simulation using machine learning hand detection. Neurosurg Focus 2023; 54 (06) E2
• 10 Tai BL, Rooney D, Stephenson F. et al. Development of a 3D-printed external ventricular drain placement simulator: technical note. J Neurosurg 2015; 123 (04) 1070-1076
• 11 Hepburn-Smith M, Dynkevich I, Spektor M, Lord A, Czeisler B, Lewis A. Establishment of an external ventricular drain best practice guideline: the quest for a comprehensive, universal standard for external ventricular drain care. J Neurosci Nurs 2016; 48 (01) 54-65
• 12 Temple CLF, Ross DC. A new, validated instrument to evaluate competency in microsurgery: the University of Western Ontario Microsurgical Skills Acquisition/Assessment instrument [outcomes article]. Plast Reconstr Surg 2011; 127 (01) 215-222
• 13 Grober ED, Hamstra SJ, Wanzel KR. et al. Validation of novel and objective measures of microsurgical skill: hand-motion analysis and stereoscopic visual acuity. Microsurgery 2003; 23 (04) 317-322
• 14 Ledwos N, Mirchi N, Yilmaz R. et al. Assessment of learning curves on a simulated neurosurgical task using metrics selected by artificial intelligence. J Neurosurg 2022; 137 (04) 1160-1171
• 15 Kahneman D. Thinking, fast and slow. Macmillan; 2011
• 16 Smit RD, Mahtabfar A, Evans JJ. Letter to the editor. How should neurosurgery approach the ethical and legal considerations for recording in the operating room?. J Neurosurg 2023; 140 (02) 606-608
• 17 Schwartz WB. Medicine and the computer. The promise and problems of change. N Engl J Med 1970; 283 (23) 1257-1264