Appl Clin Inform 2022; 13(01): 132-138
DOI: 10.1055/s-0041-1741480
Special Section on Workflow Automation

Principles for Designing and Developing a Workflow Monitoring Tool to Enable and Enhance Clinical Workflow Automation

Danny T.Y. Wu
1   Department of Biomedical Informatics, University of Cincinnati College of Medicine, Ohio, United States
2   Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, Ohio, United States
Lindsey Barrick
3   Division of Emergency Medicine, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio, United States
2   Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, Ohio, United States
Mustafa Ozkaynak
4   College of Nursing, University of Colorado-Anschutz Medical Campus, Aurora, Colorado, United States
Katherine Blondon
5   Medical and Quality Directorate, University Hospitals of Geneva, Geneva, Switzerland
6   Faculty of Medicine, University of Geneva, Geneva, Switzerland
Kai Zheng
7   Department of Informatics, University of California, Irvine, Irvine, California, United States
› Author Affiliations
Funding None.


Background Automation of health care workflows has recently become a priority. This can be enabled and enhanced by a workflow monitoring tool (WMOT).

Objectives We shared our experience in clinical workflow analysis via three cases studies in health care and summarized principles to design and develop such a WMOT.

Methods The case studies were conducted in different clinical settings with distinct goals. Each study used at least two types of workflow data to create a more comprehensive picture of work processes and identify bottlenecks, as well as quantify them. The case studies were synthesized using a data science process model with focuses on data input, analysis methods, and findings.

Results Three case studies were presented and synthesized to generate a system structure of a WMOT. When developing a WMOT, one needs to consider the following four aspects: (1) goal orientation, (2) comprehensive and resilient data collection, (3) integrated and extensible analysis, and (4) domain experts.

Discussion We encourage researchers to investigate the design and implementation of WMOTs and use the tools to create best practices to enable workflow automation and improve workflow efficiency and care quality.

Protection of Human and Animal Subjects

No human and/or animal subjects were included.

Author Contributions

The first author drafted the manuscript. All coauthors helped improve the clarity and value of the manuscript by reviewing and revising the manuscript and contributed significantly to the synthesis of the major takeaways, that is, the four principles to developing a WMOT.

Publication History

Received: 01 June 2021

Accepted: 22 November 2021

Article published online:
19 January 2022

© 2022. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

  • References

  • 1 The Office of the National Coordinator for Health Information Technology (ONC). Health information technology workflow automation policy development. Accessed May 25, 2021:
  • 2 Zayas-Cabán T, Haque SN, Kemper N. Identifying opportunities for workflow automation in health care: lessons learned from other industries. Appl Clin Inform 2021; 12 (03) 686-697
  • 3 Otokiti AU, Craven CK, Shetreat-Klein A, Cohen S, Darrow B. Beyond getting rid of stupid stuff in the electronic health record (Beyond-GROSS): protocol for a user-centered, mixed-method intervention to improve the electronic health record system. JMIR Res Protoc 2021; 10 (03) e25148
  • 4 Alam S, Osama M, Iqbal F, Sawar I. Reducing pharmacy patient waiting time. Int J Health Care Qual Assur 2018; 31 (07) 834-844
  • 5 Tsai P, Liu C, Kahler DL, Li JG, Lu B, Yan G. A self-checking treatment couch coordinate calculation system in radiotherapy. J Appl Clin Med Phys 2020; 21 (01) 43-52
  • 6 Wang M, Wang H, Xu D. The design of intelligent workflow monitoring with agent technology. Knowl Base Syst 2005; 18 (06) 257-266
  • 7 Ehrler L, Fleurke M, Purvis M, Savarimuthu BTR. Agent-based workflow management systems (WfMSs): JBees: a distributed and adaptive WfMS with monitoring and controlling capabilities. Inf Syst E-Bus Manag 2006; 4 (01) 5-23
  • 8 Holzmüller-Laue S, Schubert P, Göde B, Thurow K. Visual simulation for the bpm-based process automation. In: Kobyliński A, Sobczak A. eds. Perspectives in Business Informatics Research. Vol 158. Lecture Notes in Business Information Processing (LNBIP, volume 158). Berlin, Germany: Springer; 2013: 48-62
  • 9 Rizk Y, Isahagian V, Boag S. et al. A conversational digital assistant for intelligent process automation. In: Asatiani A, García JM, Helander N. et al., eds. Business Process Management: Blockchain and Robotic Process Automation Forum. Vol 393. Lecture Notes in Business Information Processing (LNBIP, volume 393). Switzerland: Springer International Publishing; 2020: 85-100
  • 10 Romao M, Costa J, Costa CJ. Robotic Process Automation: A Case Study in the Banking Industry. In: 2019 14th Iberian Conference on Information Systems and Technologies (CISTI). IEEE; 2019: 1-6 Coimbra, Portugal:
  • 11 Effken JA. Different lenses, improved outcomes: a new approach to the analysis and design of healthcare information systems. Int J Med Inform 2002; 65 (01) 59-74
  • 12 Carayon P, Bass E, Bellandi T, Gurses A, Hallbeck S, Mollo V. Socio-technical systems analysis in health care: a research agenda. IIE Trans Healthc Syst Eng 2011; 1 (01) 145-160
  • 13 Harrison MI, Koppel R, Bar-Lev S. Unintended consequences of information technologies in health care–an interactive sociotechnical analysis. J Am Med Inform Assoc 2007; 14 (05) 542-549
  • 14 Bloomrosen M, Starren J, Lorenzi NM, Ash JS, Patel VL, Shortliffe EH. Anticipating and addressing the unintended consequences of health IT and policy: a report from the AMIA 2009 Health Policy Meeting. J Am Med Inform Assoc 2011; 18 (01) 82-90
  • 15 Cabitza F, Rasoini R, Gensini GF. Unintended consequences of machine learning in medicine. JAMA 2017; 318 (06) 517-518
  • 16 Ogundaini O, de la Harpe R, McLean N. Unintended consequences of technology-enabled work activities experienced by healthcare professionals in tertiary hospitals of sub-Saharan Africa. Afr J Sci Technol Innov Dev 2021; 1-10
  • 17 Vankipuram M, Kahol K, Cohen T, Patel VL. Toward automated workflow analysis and visualization in clinical environments. J Biomed Inform 2011; 44 (03) 432-440
  • 18 Zhu Q, Nie H, Lu X, Duan H. Radiology workflow-based monitoring dashboard in a heterogeneous environment. In: 2010 3rd International Conference on Biomedical Engineering and Informatics. IEEE; 2010: 2494-2498 Yantai, China:
  • 19 Patel VL, Denton CA, Soni HC, Kannampallil TG, Traub SJ, Shapiro JS. Physician workflow in two distinctive emergency departments: an observational study. Appl Clin Inform 2021; 12 (01) 141-152
  • 20 Sieja A, Kim E, Holmstrom H. et al. Multidisciplinary sprint program achieved specialty-specific EHR optimization in 20 clinics. Appl Clin Inform 2021; 12 (02) 329-339
  • 21 Yu D, Obuseh M, DeLaurentis P. Quantifying the impact of infusion alerts and alarms on nursing workflows: a retrospective analysis. Appl Clin Inform 2021; 12 (03) 528-538
  • 22 Gold JA, Becton J, Ash JS, Corby S, Mohan V. Do you know what your scribe did last spring? The impact of COVID-19 on medical scribe workflow. Appl Clin Inform 2020; 11 (05) 807-811
  • 23 Zheng K, Guo MH, Hanauer DA. Using the time and motion method to study clinical work processes and workflow: methodological inconsistencies and a call for standardized research. J Am Med Inform Assoc 2011; 18 (05) 704-710
  • 24 Zheng K, Ciemins E, Lanham H, Lindberg C. Examining the relationship between health IT and ambulatory care workflow redesign. . Accessed December 10, 2021:
  • 25 Zheng K, Westbrook J, Kannampallil TG, Patel VL. Cognitive Informatics: Reengineering Clinical Workflow for Safer and More Efficient Care. Switzerland: Springer Nature; 2019
  • 26 O'Neil C, Schutt R. Doing Data Science. 1st ed.. Newton MA: O'Reilly Media Inc.; 2013
  • 27 Ozkaynak M, Wu DTY, Hannah K, Dayan PS, Mistry RD. Examining workflow in a pediatric emergency department to develop a clinical decision support for an antimicrobial stewardship program. Appl Clin Inform 2018; 9 (02) 248-260
  • 28 Barrick L, Wu DTY, Frey T. et al. Improving care delivery: location timestamps to enhance process measurement of a clinical workflow. Pediatr Qual Saf 2021; 6 (05) e475
  • 29 Hribar MR, Read-Brown S, Goldstein IH. et al. Secondary use of electronic health record data for clinical workflow analysis. J Am Med Inform Assoc 2018; 25 (01) 40-46
  • 30 Adler-Milstein J, Zhao W, Willard-Grace R, Knox M, Grumbach K. Electronic health records and burnout: time spent on the electronic health record after hours and message volume associated with exhaustion but not with cynicism among primary care clinicians. J Am Med Inform Assoc 2020; 27 (04) 531-538
  • 31 Overmann KM, Wu DTY, Xu CT, Bindhu SS, Barrick L. Real-time locating systems to improve healthcare delivery: a systematic review. J Am Med Inform Assoc 2021; 28 (06) 1308-1317
  • 32 Unertl KM, Novak LL, Johnson KB, Lorenzi NM. Traversing the many paths of workflow research: developing a conceptual framework of workflow terminology through a systematic literature review. J Am Med Inform Assoc 2010; 17 (03) 265-273
  • 33 Maguire M, Delahunt B. Doing a thematic analysis: a practical, step-by-step guide for learning and teaching scholars. All Irel J High Educ 2017;9(03):
  • 34 Mei Q, Cai D, Zhang D, Zhai C. Topic modeling with network regularization. In: Proceeding of the 17th International Conference on World Wide Web - WWW '08. ACM Press; 2008: 101
  • 35 Zheng K, Haftel HM, Hirschl RB, O'Reilly M, Hanauer DA. Quantifying the impact of health IT implementations on clinical workflow: a new methodological perspective. J Am Med Inform Assoc 2010; 17 (04) 454-461
  • 36 Sinsky CA, Rule A, Cohen G. et al. Metrics for assessing physician activity using electronic health record log data. J Am Med Inform Assoc 2020; 27 (04) 639-643
  • 37 Vankipuram A, Traub S, Patel VL. A method for the analysis and visualization of clinical workflow in dynamic environments. J Biomed Inform 2018; 79: 20-31
  • 38 User-centered design basics. . Accessed December 10, 2021:
  • 39 Staras S, Tauscher JS, Rich N. et al. Using a clinical workflow analysis to enhance eHealth implementation planning: tutorial and case study. JMIR Mhealth Uhealth 2021; 9 (03) e18534
  • 40 Sills MR, Ozkaynak M, Jang H. Predicting hospitalization of pediatric asthma patients in emergency departments using machine learning. Int J Med Inform 2021; 151: 104468
  • 41 Tanzini M, Westbrook JI, Guidi S, Sunderland N, Prgomet M. Measuring Clinical Workflow to Improve Quality and Safety. In: Donaldson L, Ricciardi W, Sheridan S, Tartaglia R. eds. Textbook of Patient Safety and Clinical Risk Management. Switzerland: Springer International Publishing; 2021: 393-402