NextLens—The Next Generation of Surgical Navigation: Proof of Concept of an Augmented Reality System for Surgical Navigation

Abstract

Objective The aim of this work was the development of an augmented reality system including the functionality of conventional surgical navigation systems.

Methods An application software for the Augmented Reality System HoloLens 2 from Microsoft was developed. It detects the position of the patient as well as position of surgical instruments in real time and displays it within the two-dimensional (2D) magnetic resonance imaging or computed tomography (CT) images. The surgical pointer instrument, including a pattern that is recognized by the HoloLens 2 sensors, was created with three-dimensional (3D) printing. The technical concept was demonstrated at a cadaver skull to identify anatomical landmarks.

Results With the help of the HoloLens 2 and its sensors, the real-time position of the surgical pointer instrument could be shown. The position of the 3D-printed pointer with colored pattern could be recognized within 2D-CT images when stationary and in motion at a cadaver skull. Feasibility could be demonstrated for the clinical application of transsphenoidal pituitary surgery.

Conclusion The HoloLens 2 has a high potential for use as a surgical navigation system. With subsequent studies, a further accuracy evaluation will be performed receiving valid data for comparison with conventional surgical navigation systems. In addition to transsphenoidal pituitary surgery, it could be also applied for other surgical disciplines.

Keywords

augmented reality - surgical navigation - transsphenoidal pituitary surgery - computer-assisted surgery

Authors' Contributions

R.G., D.W., C.S.: technical idea, writing of manuscript; M.B., L.A.: software instrument tracking; P.G.: design of surgical pointer, review of manuscript; F.K.: segmentation CT, construction of surgical pointer; E.G., R.M., S.S.: review of manuscript.

Ethics Approval

No ethical approval was required. The cadaver skull was applied according to the Declaration of Helsinki.

Consent to Participate

The cadaver skull was applied according to the declaration of Helsinki.

Data Availability Statement

The data that support the findings of this study are available from the corresponding author, upon request.

Publication History

Received: 16 February 2023

Accepted: 24 April 2023

Accepted Manuscript online:
28 April 2023

Article published online:
01 June 2023

• References

• 1 Amundson M, Newman M, Cheng A, Khatib B, Cuddy K, Patel A. Three-dimensional computer-assisted surgical planning, manufacturing, intraoperative navigation, and computed tomography in maxillofacial trauma. Atlas Oral Maxillofac Surg Clin North Am 2020; 28 (02) 119-127
• 2 Bergeron L, Bouchard S, Bonapace-Potvin M, Bergeron F. Reply: intraoperative surgical navigation reduces the surgical time required to treat acute major facial fractures. Plast Reconstr Surg 2020; 146 (04) 509e-510e
• 3 Demian N, Pearl C, Woernley III TC, Wilson J, Seaman J. Surgical navigation for oral and maxillofacial surgery. Oral Maxillofac Surg Clin North Am 2019; 31 (04) 531-538
• 4 Kim Y, Lee BH, Mekuria K. et al. Registration accuracy enhancement of a surgical navigation system for anterior cruciate ligament reconstruction: a phantom and cadaveric study. Knee 2017; 24 (02) 329-339
• 5 Hung K-F, Wang F, Wang H-W, Zhou W-J, Huang W, Wu Y-Q. Accuracy of a real-time surgical navigation system for the placement of quad zygomatic implants in the severe atrophic maxilla: a pilot clinical study. Clin Implant Dent Relat Res 2017; 19 (03) 458-465
• 6 Deng W, Li F, Wang M, Song Z. Easy-to-use augmented reality neuronavigation using a wireless tablet PC. Stereotact Funct Neurosurg 2014; 92 (01) 17-24
• 7 Mehbodniya AH, Moghavvemi M, Narayanan V, Waran V. Frequency and causes of line of sight issues during neurosurgical procedures using optical image-guided systems. World Neurosurg 2019; 122: e449-e454
• 8 Ewurum CH, Guo Y, Pagnha S, Feng Z, Luo X. Surgical navigation in orthopedics: workflow and system review. Adv Exp Med Biol 2018; 1093: 47-63
• 9 Tschopp KP, Thomaser EG. Outcome of functional endonasal sinus surgery with and without CT-navigation. Rhinology 2008; 46 (02) 116-120
• 10 Briem D, Linhart W, Lehmann W. et al. Computer-assisted screw insertion into the first sacral vertebra using a three-dimensional image intensifier: results of a controlled experimental investigation. Eur Spine J 2006; 15 (06) 757-763
• 11 Akca F, Schwagten B, Theuns DAJ, Takens M, Musters P, Szili-Torok T. Safety and feasibility of single-catheter ablation using remote magnetic navigation for treatment of slow-fast atrioventricular nodal reentrant tachycardia compared to conventional ablation strategies. Acta Cardiol 2013; 68 (06) 559-567
• 12 Frantz T, Jansen B, Duerinck J, Vandemeulebroucke J. Augmenting Microsoft's HoloLens with Vuforia tracking for neuronavigation. Healthc Technol Lett 2018; 5 (05) 221-225
• 13 Scherl C, Stratemeier J, Karle C. et al. Augmented reality with HoloLens in parotid surgery: how to assess and to improve accuracy. Eur Arch Otorhinolaryngology 2021; 278 (07) 2473-2483
• 14 Shi L, Luo T, Zhang L. et al. Preliminary use of HoloLens glasses in surgery of liver cancer. Zhong Nan Da Xue Xue Bao Yi Xue Ban 2018; 43 (05) 500-504
• 15 Tepper OM, Rudy HL, Lefkowitz A. et al. Mixed reality with HoloLens: where virtual reality meets augmented reality in the operating room. Plast Reconstr Surg 2017; 140 (05) 1066-1070
• 16 Koutourousiou M, Gardner PA, Fernandez-Miranda JC, Paluzzi A, Wang EW, Snyderman CH. Endoscopic endonasal surgery for giant pituitary adenomas: advantages and limitations. J Neurosurg 2013; 118 (03) 621-631
• 17 Zwagerman NT, Zenonos G, Lieber S. et al. Endoscopic transnasal skull base surgery: pushing the boundaries. J Neurooncol 2016; 130 (02) 319-330
• 18 Grunert R, Klietz S, Gardner PA, Fernandez-Miranda JC, Snyderman CH. Evaluation of bendable surgical suction devices made of shape-memory alloy for the endonasal transsphenoid removal of pituitary tumors. Ear Nose Throat J 2018; 97 (12) 413-416
• 19 Thabit A, Niessen WJ, Wolvius EB, van Walsum T. Evaluation of marker tracking using mono and stereo vision in Microsoft HoloLens for surgical navigation. In:SPIE Proceedings Volume 12034, Medical Imaging 2022: Image-Guided Procedures, Robotic Interventions, and Modeling. San Diego, California, United States; 2022
• 20 Schuppstuhl T, Tracht K, Henrich D. Annals of Scientific Society for Assembly, Handling and Industrial Robotics. 1st ed. 2020. Berlin, Heidelberg: MHI e.V; 2020
• 21 Pérez-Pachón L, Sharma P, Brech H. et al. Effect of marker position and size on the registration accuracy of HoloLens in a non-clinical setting with implications for high-precision surgical tasks. Int J CARS 2021; 16 (06) 955-966
• 22 van Doormaal TPC, van Doormaal JAM, Mensink T. Clinical accuracy of holographic navigation using point-based registration on augmented-reality glasses. Oper Neurosurg (Hagerstown) 2019; 17 (06) 588-593
• 23 Zhou Z, Yang Z, Jiang S, Zhuo J, Zhu T, Ma S. Surgical navigation system for hypertensive intracerebral hemorrhage based on mixed reality. J Digit Imaging 2022; 35 (06) 1530-1543