Evaluating Usability of a Touchless Image Viewer in the Operating Room

Abstract

Background Availability of patient-specific image data, gathered from preoperatively conducted studies, like computed tomography scans and magnetic resonance imaging studies, during a surgical procedure is a key factor for surgical success and patient safety. Several alternative input methods, including recognition of hand gestures, have been proposed for surgeons to interact with medical image viewers during an operation. Previous studies pointed out the need for usability evaluation of these systems.

Objectives We describe the accuracy and usability of a novel software system, which integrates gesture recognition via machine learning into an established image viewer.

Methods This pilot study is a prospective, observational trial, which asked surgeons to interact with software to perform two standardized tasks in a sterile environment, modeled closely to a real-life situation in an operating room. To assess usability, the validated “System Usability Scale” (SUS) was used. On a technical level, we also evaluated the accuracy of the underlying neural network.

Results The neural network reached 98.94% accuracy while predicting the gestures during validation. Eight surgeons with an average of 6.5 years of experience participated in the usability study. The system was rated on average with 80.25 points on the SUS.

Conclusion The system showed good overall usability; however, additional areas of potential improvement were identified and further usability studies are needed. Because the system uses standard PC hardware, it made for easy integration into the operating room.

Keywords

orthopaedics - machine learning - neural networks - gestures - operating rooms

Protection of Human and Animal Subjects

All procedures performed in this study involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. This article does not contain any studies with animals performed by any of the authors. Informed consent was obtained from all individual participants included in the study.

• References

• 1 WHO Guidelines for Safe Surgery 2009: Safe Surgery Saves Lives. Geneva: World Health Organization; 2009. . Available at: http://www.ncbi.nlm.nih.gov/books/NBK143243/ . Accessed August 6, 2019
• 2 Grimm BD, Laxer EB, Blessinger BJ, Rhyne AL, Darden BV. Wrong-level spine surgery. JBJS Rev 2014; 2 (03) 1
• 3 Schultz M, Gill J, Zubairi S, Huber R, Gordin F. Bacterial contamination of computer keyboards in a teaching hospital. Infect Control Hosp Epidemiol 2003; 24 (04) 302-303
• 4 Cho Y, Lee A, Park J, Ko B, Kim N. Enhancement of gesture recognition for contactless interface using a personalized classifier in the operating room. Comput Methods Programs Biomed 2018; 161: 39-44
• 5 O'Hara K, Dastur N, Carrell T. , et al. Touchless interaction in surgery. Commun ACM 2014; 57 (01) 70-77
• 6 Mewes A, Hensen B, Wacker F, Hansen C. Touchless interaction with software in interventional radiology and surgery: a systematic literature review. Int J CARS 2017; 12 (02) 291-305
• 7 Wipfli R, Dubois-Ferrière V, Budry S, Hoffmeyer P, Lovis C. Gesture-controlled image management for operating room: a randomized crossover study to compare interaction using gestures, mouse, and third person relaying. PLoS One 2016; 11 (04) e0153596
• 8 Alvarez-Lopez F, Maina MF, Saigí-Rubió F. Use of commercial off-the-shelf devices for the detection of manual gestures in surgery: systematic literature review. J Med Internet Res 2019; 21 (05) e11925
• 9 Strickland M, Tremaine J, Brigley G, Law C. Using a depth-sensing infrared camera system to access and manipulate medical imaging from within the sterile operating field. Can J Surg 2013; 56 (03) E1-E6
• 10 Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. ArXiv14091556 Cs. September 2014. Available at: http://arxiv.org/abs/1409.1556 . Accessed August 15, 2019
• 11 Russakovsky O, Deng J, Su H. , et al. ImageNet large scale visual recognition challenge. ArXiv14090575 Cs. September 2014. Available at: http://arxiv.org/abs/1409.0575 . Accessed August 17, 2019
• 12 Tan JH, Chao C, Zawaideh M, Roberts AC, Kinney TB. Informatics in radiology: developing a touchless user interface for intraoperative image control during interventional radiology procedures. Radiographics 2013; 33 (02) E61-E70
• 13 Jacob MG, Wachs JP, Packer RA. Hand-gesture-based sterile interface for the operating room using contextual cues for the navigation of radiological images. J Am Med Inform Assoc 2013; 20 (e1): e183-e186
• 14 Ebert LC, Hatch G, Thali MJ, Ross S. Invisible touch—control of a DICOM viewer with finger gestures using the Kinect depth camera. J Forensic Radiol Imaging 2013; 1 (01) 10-14
• 15 Rossol N, Cheng I, Shen S, Basu A. Touchfree medical interfaces. Conf Proc IEEE Eng Med Biol Soc 2014; 2014: 6597-6600
• 16 Brooke J. SUS-a quick and dirty usability scale. In: Usability Evaluation in Industry. London, Bristol, PA: Taylor & Francis; 1996: 189-194
• 17 Sousa VEC, Dunn Lopez K. Towards usable E-Health. A systematic review of usability questionnaires. Appl Clin Inform 2017; 8 (02) 470-490
• 18 Brooke J. SUS: a retrospective. J Usability Stud 2013; 8 (02) 29-40
• 19 Rummel B. System Usability Scale – jetzt auch auf Deutsch. SAP User Experience Community. Published January 13, 2015. Available at: https://experience.sap.com/skillup/system-usability-scale-jetzt-auch-auf-deutsch/ . Accessed March 8, 2019
• 20 Bangor A, Kortum PT, Miller JT. An empirical evaluation of the System Usability Scale. Int J Hum Comput Interact 2008; 24 (06) 574-594
• 21 Hultman G, Marquard J, Arsoniadis E. , et al. Usability testing of two ambulatory EHR navigators. Appl Clin Inform 2016; 7 (02) 502-515
• 22 Staggers N, Xiao Y, Chapman L. Debunking health IT usability myths. Appl Clin Inform 2013; 4 (02) 241-250
• 23 Chaudhry A, Sutton C, Wood J, Stone R, McCloy R. Learning rate for laparoscopic surgical skills on MIST VR, a virtual reality simulator: quality of human-computer interface. Ann R Coll Surg Engl 1999; 81 (04) 281-286