Registered Nurse Strain Detection Using Ambient Data: An Exploratory Study of Underutilized Operational Data Streams in the Hospital Workplace

Abstract

Background Registered nurses (RNs) regularly adapt their work to ever-changing situations but routine adaptation transforms into RN strain when service demand exceeds staff capacity and patients are at risk of missed or delayed care. Dynamic monitoring of RN strain could identify when intervention is needed, but comprehensive views of RN work demands are not readily available. Electronic care delivery tools such as nurse call systems produce ambient data that illuminate workplace activity, but little is known about the ability of these data to predict RN strain.

Objectives The purpose of this study was to assess the utility of ambient workplace data, defined as time-stamped transaction records and log file data produced by non-electronic health record care delivery tools (e.g., nurse call systems, communication devices), as an information channel for automated sensing of RN strain.

Methods In this exploratory retrospective study, ambient data for a 1-year time period were exported from electronic nurse call, medication dispensing, time and attendance, and staff communication systems. Feature sets were derived from these data for supervised machine learning models that classified work shifts by unplanned overtime. Models for three timeframes —8, 10, and 12 hours—were created to assess each model's ability to predict unplanned overtime at various points across the work shift.

Results Classification accuracy ranged from 57 to 64% across three analysis timeframes. Accuracy was lowest at 10 hours and highest at shift end. Features with the highest importance include minutes spent using a communication device and percent of medications delivered via a syringe.

Conclusion Ambient data streams can serve as information channels that contain signals related to unplanned overtime as a proxy indicator of RN strain as early as 8 hours into a work shift. This study represents an initial step toward enhanced detection of RN strain and proactive prevention of missed or delayed patient care.

Keywords

machine learning - workplace - nursing - informatics - secondary analysis - stress - real time - health system

Background and Significance

Effective management of fluctuating demand is a well-recognized challenge in hospital work systems. High workload and resource insufficiency are associated with adverse patient events,[1] errors of omission,[2] job dissatisfaction,[3] and increased patient mortality.[4] Employees who repeatedly overextend themselves to sustain high-quality care are at increased risk of fatigue and burnout.[5] [6] [7] Although burnout is a recognized risk among registered nurses (RNs),[8] financial pressures compel hospitals to maximize the number of patients served without increasing payroll.[9] Hospital leaders find themselves in search of pragmatic solutions to manage competing priorities of cost, clinician work experience, patient experience, and outcomes.[10] In pursuit of such solutions, there is a need for early recognition of adverse workplace conditions[11] that can negatively impact clinician work experience and care quality.

Intensive care unit (ICU) capacity strain has been defined as a temporally varying influence on a given ICU's ability to provide high-quality care for persons who are or could become a patient in an ICU on a given day.[12] In the current study, we explore RN strain at the work shift level as an aspect of the larger concept of ICU capacity strain. Multiple terms have been used to describe relationships between supply–demand imbalance and negative patient or staff outcomes. These terms include work intensification,[13] operational failures,[14] missed nursing care,[15] workplace decompensation,[11] and swamping.[16] For the purposes of this study, RN strain is defined as temporal phenomenon in which service demand exceeds RN capacity, necessitating intervention to avoid degradation in quality of patient care or RN work experience.

Hospitals have limited informatics support for dynamic sensing of adverse working conditions including periods of supply–demand imbalance at the bedside. Current electronic health record (EHR)-based workload estimation tools are designed to inform staffing decisions[17] rather than to detect focused periods of supply–demand imbalance that may necessitate tradeoffs between thoroughness and efficiency.[18] Early signs of work system compromise often appear as small changes in facility operations,[19] and hospitals primarily rely on charge nurses to recognize these changes as they emerge. Care delivery tools including electronic communication, nurse call, and medication dispensing systems automatically produce ambient data that reflect workplace activity, but little is known about relationships between RN strain and these non-EHR data.

Purpose

The purpose of this exploratory retrospective study was to assess the utility of ambient workplace data, defined as time-stamped transactional records and log file data that are automatically produced by non-EHR care delivery tools, as an information channel for automated sensing of RN strain during hospital work shifts. Although the EHR is a prominent electronic care delivery tool, descriptions of care activity in the EHR typically require intentional human documentation or manual verification of interfaced data. An example of dynamically generated ambient data is automated production of a date–time-stamped record when a patient presses a call light to request assistance, and when a staff person responds to the bedside. We hypothesize that features derived from ambient data, such as call light duration, may differ by level of RN strain, and that future dynamic sensing of activity patterns associated with RN strain may enhance a patient care unit's ability to sense and proactively respond to strain in the hospital workplace.

Methods

Setting

The study facility was a 16-bed adult medical ICU (MICU) in a large academic medical center in the Pacific Northwest. RNs on this unit work 12-hour shifts. Bedside RNs carry primary responsibility for patient surveillance and delivery of bedside care for up to two assigned patients. The patient care unit runs at over 85% occupancy and employs 70 full- and part-time employees. The number of on-duty RNs in each work shift varies by patient census and severity of illness. Additional nurses float to this unit from other ICUs and a critical care float pool. In addition to bedside RNs, daytime work shifts were staffed with a charge nurse who manages patient flow and administrative functions. Most shifts were also staffed with a relief nurse who serves as a resource for clinical emergencies and other care delivery needs. The unit utilizes health assistants for supply restocking but does not routinely use nursing assistants in care delivery. The physical unit is laid out in an H-shape with patient care rooms lining the building exterior and staff workspaces between interior hallways.

Data Sources

In a separately reported study, RNs described patterns of workplace activity that tend to change during times of strain.[20] Focus group findings informed the selection of data sources for this study, which include ambient data produced by Omnicell automated medication dispensing cabinets, a Kronos time and attendance system, a Rauland nurse call system, and a Vocera electronic communication system. Omnicell data reflect activity related to physical acquisition of medications for subsequent administration to patients. Kronos data reflect the department in which staff worked on a given calendar day and staff hours attributable to regular, overtime, and other types of pay codes. Rauland data reflect patient and staff's use of bed- and room-integrated call lights used to summon assistance. Vocera data reflect staff-to-staff communication via a wearable, voice-activated device.

Common characteristics of selected data sources include automated production as a byproduct of tool use rather than intentional human documentation, reflection of both the timing and type of activity that occurred, and relationship to strain-related patterns of activity as described by nurses in a separately reported focus group study.[20] Data attributable to the study unit, nursing staff, or patients during the year 2016 were exported by hospital system administrators using back-end reporting tools for each electronic system.

Measurement of RN Strain

Because RNs do not record subjective ratings of RN strain each work shift, a proxy indicator is needed to estimate RN strain. EHR-generated patient acuity scores are a candidate proxy indicator, but the study site did not utilize EHR-based patient acuity or work intensity tools at the time of this study. Instead, this study utilized a clinical expert panel to recommend a proxy indicator of strain. The study's clinical expert panel was composed of clinical leaders who were employed at the study facility, possessed a detailed understanding of bedside care delivery, and had previous care improvement experience. The panel consisted of eight nursing, physician, and hospital information technology representatives representing two adult patient care units, clinical quality, nursing administration, process improvement, internal medicine, and clinical informatics departments. The clinical expert panel recommended use of unplanned overtime as a proxy indicator of RN strain. The panel defined unplanned overtime as RN overtime greater than 30 minutes but less than 3 hours. The panel's rationale was that overtime greater than 30 minutes but less than 3 hours is common on work shifts where an RN experienced high workload and stayed late to complete his or her work. The panel noted that the overtime duration of 30 minutes or less occurs for many reasons other than RN strain, and overtime greater than 3 hours is often associated with planned overtime, such as RN coverage of an absence. The overtime duration was derived from time and attendance system data. Each work shift was labeled “unplanned overtime present” if one or more RNs logged between 30 minutes and 3 hours of overtime, and “unplanned overtime absent” if all on-duty RNs logged no more than 30 minutes of overtime.

Data Preparation

The daytime work shift was selected as the unit of analysis in this study to facilitate analysis of shifts with similar RN work responsibilities and temporal rhythms. As is common in informatics studies that combine disparate datasets, data preparation was a multistep, iterative process. Multiple date–time formats were normalized across data sources. Raw ambient data, consisting of time-stamped transactional records and log file data generated through workplace use of a communication device, nurse call system, time and attendance system, and medication dispensing cabinet, were mapped to 12-hour work shifts based on associated date–time stamps. Overtime data, available for export by person and calendar date rather than by person, calendar date, and work shift, was associated with a day or night shift based on the time of day RNs dispensed medications from automated dispensing cabinets.

Specifications of filter criteria for removal of extraneous data were established through discussion with the study's clinical expert panel. For example, raw medication dispensing data were filtered to “issued medication” events to reflect activity related to bedside patient care rather than cabinet restocking. Vocera communication data were filtered to communication events that occurred on shifts in which the associated RN worked on the study unit, as determined by time and attendance and medication administration records. Rauland nurse call data included a variable number of subevents (e.g., call initiated, escalated, responded, cancelled) depending on the number of call escalations that occurred, whether a staff person spoke to the patient before a nurse physically entered the room, and the manner in which the call event was terminated. Subevent records were aggregated into a consolidated record reflecting the start, end, and duration of each nurse call event. Similarly, multiple Kronos time and attendance records (e.g., regular time, differential time, overtime) were consolidated into a single time and attendance record reflecting the start time, end time, and overtime minutes associated with each RN person shift. In the absence of a common identifier, RN identities were linked across systems through automated text matching augmented by manual reconciliation of names.

Data Modeling

A range of summary models, known as features, was generated to summarize digitally observable care activity for each daytime work shift. Features transform raw date–time stamps into variables that characterize workplace activity. Feature definition was guided by findings from a separately reported[20] focus group study in which frontline RNs described patterns of activity that tend to emerge during times of strain. For example, RNs described that response to bedside nurse call lights is often delayed during periods of strain. This domain insight guided definitions of the sum of active nurse call light minutes and count of nurse call events of greater than 3 minutes duration as features in this study.

A set of features representing nurse call, communication, and medication-related workplace activity was constructed for three analysis timeframes for each work shift: shift hours 0 to 8 (7 a.m. to 3 p.m.), shift hours 0 to 10 (7 a.m. to 5 p.m.), and the full 12-hour shift (7 a.m. to 7 p.m.). Feature definitions were consistent across analysis timeframes. Feature values were calculated using ambient data generated during corresponding timeframes. Assessment of the presence of strain-related signal at multiple time points is important because in future application of this research to practice, charge nurses will pragmatically require strain-related insight during a shift to facilitate proactive intervention before shift end.

Supervised machine learning, a method for associating patterns in data with known outcomes, was used to generate binary classification models for each analysis timeframe using time-correspondent feature sets.[21] Five machine learning algorithms were applied to 50 candidate features to assess the utility of ambient workplace data as an information channel for automated sensing of unplanned overtime, as a proxy measure of RN strain during hospital work shifts.

Automated feature selection was performed using the Python scikit-learn recursive feature elimination with cross-validation (RFECV) algorithm.[22] Supervised eightfold cross-validation training and testing was performed for the Extra Decision Tree, Gradient Boost, Random Forest, Logistic Regression, and Support Vector Machine algorithms,[21] created for each analysis timeframe. In the training process, feature set data from 7/8 of included work shifts were used to create supervised machine learning models, which were tested against the remaining 1/8 of feature set data. This training and testing process was repeated eight times. Overall accuracy for each model was assessed using the scikit-learn balanced accuracy scoring metric.

Results

A total of 110,649 time-stamped ambient data records were exported from two medication dispensing cabinets, a time and attendance system, a bedside nurse call system, and an electronic communication system, representing the activity of 144 regular, part-time, or float RNs who worked one or more shifts on the study unit and participated in the care of approximately 1,400 MICU patients during the year 2016 ([Table 1]). The mean number of on-duty RNs was 14, including a charge nurse and a relief nurse. Midnight average daily census was 13.8 patients on the 16-bed MICU study unit.

Table 1 Raw ambient data row count by operational information system

Information system	Event records
Medication dispensing cabinet: Omnicell	49,595
Time and attendance system: Kronos	11,935
Bedside nurse call system: Rauland	14,527
Electronic communication system: Vocera	34,592
Total	110,649

Of 366 daytime work shifts in 2016, 191 shifts included overtime between 30 minutes and 3 hours by one or more nurses, which met the study criteria for presence of unplanned overtime. Shifts with unplanned overtime were distributed evenly throughout the calendar year.

The central tendency and spread of shift-level feature values across 191 work shifts with and 175 work shifts without unplanned overtime are summarized in [Table 2]. The change in direction of features, such as longer patient wait times for response to patient-initiated call lights on shifts with unplanned overtime, was largely consistent with separately reported[20] RN descriptions of activity patterns that emerge during times of strain ([Table 2]).

Table 2 Central tendency and spread of shift-level feature values by outcome state

Outcome	Overtime absent (175 shifts)		Overtime present (191 shifts)
Shift-level features	Mean	SD	Mean	SD
Features selected in one or models
Sum of communication device minutes	25.7	11.1	31.4	13.0
Skewness of medication count across RNs	0.3	0.8	0.2	0.6
Percent of medications delivered via syringe	26.8%	0.1	29.1%	0.1
Sum of medications dispensed in shift hours 1–3	43.9	15.3	46.6	14.5
Sum of nurse call minutes (patient wait time)	27.3	15.4	30.6	16.4
Percent of patients present on the previous calendar day (patient continuity)	72.6%	0.1	70.0%	0.1
Mean medication dispensing count per patient	9.5	1.8	9.8	1.8
Example features not selected in any model
Count of cross-assignment medication dispenses	7.6	3.5	9.1	3.7
Sum of medication dispenses across all patients	129.6	31.5	140.4	31.5
Aggregate years of RN experience	71.4	19.8	75.1	19.1
Percent RNs who worked the previous calendar day (RN continuity)	31.1%	0.1	30.6%	0.1
Count of calls to predefined groups (e.g., lift team)	4.6	2.4	5.0	2.7

Abbreviation: RN, registered nurse.

Of five machine learning models generated for each analysis timeframe, Gradient Boost consistently provided the highest classification accuracy. Across analysis timeframes, Gradient Boost machine learning models were composed of three to eight features, selected through application of a recursive feature elimination algorithm. Feature selection results and machine learning performance metrics are summarized in [Table 3]. Overall accuracy of classification of work shifts by the presence or absence of unplanned overtime ranged from 57% at shift hour 10 to 64% at shift hour 12.

Table 3 Feature selection and performance for three supervised Gradient Boost models

Gradient Boost models	Selected feature sets for each analysis timeframe, listed in order of decreasing importance	Accuracy	F1	AUC
Shift hour 8 Features based on data produced between 7:00 a.m. and 3:00 p.m.	1. Sum of communication device minutes 2. Skewness of medication count across RNs 3. Percent of medications delivered via syringe 4. Sum of medications dispensed in shift hours 1–3	0.599	0.597	0.596
Shift hour 10 Features based on data produced between 7:00 a.m. and 5:00 p.m.	1. Sum of communication device minutes 2. Percent of medications delivered via syringe 3. Sum of nurse call minutes (patient wait time)	0.572	0.571
Shift hour 12 Features based on data produced between 7:00 a.m. and 7:00 p.m.	1. Sum of communication device minutes 2. Percent of medications delivered via syringe 3. Skewness of medication count across RNs 4. Sum of nurse call minutes (patient wait time) 5. Sum of medications dispensed in shift hours 1–3 6. Percent of patients present on the previous calendar day (patient continuity) 7. Mean medication count per patient 8. Skewness of medication count across patients	0.640	0.639	0.638

Abbreviations: AUC, area under the curve; RN, registered nurse.

Features with the highest importance across all three analysis timeframes include minutes spent using a communication device and percent of medications delivered via a syringe ([Table 4]). Skewness of medication count across RNs, patient wait time for bedside assistance, and the sum of medications dispensed between 7:00 and 10:00 a.m. were selected in at least two of three machine learning models.

Table 4 Features selected in more than one machine learning model

Feature description	Shift hour 8	Shift hour 10	Shift hour 12
Sum of communication device minutes	x
Percent of medications delivered via syringe	x
Skewness of medication count across RNs	x		x
Sum of nurse call minutes (patient wait time)		x
Sum of medications dispensed in shift hours 1–3	x		x

Abbreviation: RN, registered nurse.

Discussion

This exploratory study demonstrated feasibility of secondary analysis of ambient workplace data and highlighted quantifiable features that are associated with unplanned overtime as a proxy indicator of strain. A key finding of this study was that operational data streams can serve as information channels that contain signals related to unplanned overtime as early as 8 hours into a work shift. The feature with greatest importance across all analysis timeframes was the length of time RNs spent using an electronic communication device. A comparison of this feature across shifts reveals that mean time spent in use of a communication device is higher on shifts with unplanned overtime than on shifts without. Staff-to-staff communication is required during workplace events such as management of patient deterioration and transfer of patients in and out of the patient care unit, and these events are known contributors to workload.[23] [24] [25] Effectiveness of communication between hospital personnel has previously been recognized as a factor in missed nursing care,[26] which may also explain the importance this communication-related feature in this study.

Medication administration is a prominent nursing activity and unsurprisingly, medication-related features were important in all classification models. Difference in the quantity of medications delivered via a syringe across shifts with and without unplanned overtime may be explained by conventional wisdom that intravenous and intramuscular medications are used in times of crises and with high acuity patients because they are fast acting. Skewness of medications across RNs can occur when assignments are uneven, or an RN has a particularly unstable patient. The number of medications dispensed in the first 3 hours of the shift includes a typical surge in medication administration related to 9:00 a.m. daily medications. Shifts with overtime had a higher mean medication administration than shifts without, which is consistent with nurses' qualitative report that if an RN falls behind during morning hours, the pace of work can be negatively impacted for the entire work shift.[20]

The sum of nurse call minutes, representing the aggregate time patient call lights were active across patient rooms during a shift, was also an important feature. Nurses juggle multiple concurrent priorities and dynamically prioritize work to attend to work that has the highest clinical priority.[27] Addressing urgent needs before routine nurse call lights is an example tradeoff between thoroughness and efficiency[18] and is consistent with nurse descriptions of nearly continuous audible call light tones during times of strain.[20]

A second key finding was that strain-related signals within ambient data streams have potential to augment human observation of the workplace. Through focus group interviews, frontline nurses explained that demand bursts as short as 20 minutes can produce care delays that can last twice the length of the original burst.[20] Experienced nurses routinely use clinical foresight and thinking-in-action skills[28] to buffer variable demand but workload and patient safety can become compromised once nurses' adaptive capacity is exhausted.[29] Previous studies establish associations between time scarcity, missed care, and negative patient outcomes,[4] [30] [31] but automatically generated ambient data streams exist in separate data systems at the study facility, limiting their use in daily workplace monitoring. Hospitals have limited informatics support for dynamic recognition of escalating RN strain, nurse fatigue,[32] and workload factors not observable in the EHR, even though these factors can threaten the timeliness or completeness of patient care.

We assert that decision support for early recognition of demand bursts during a work shift is as important as support for staffing decisions at the beginning of a shift. Clinical and operational challenges produce variable-length periods of high time and production pressure throughout a work shift. The impact of variable demand can be intensified by cultural aspects of a work system including a tendency to avoid open discussion of problems[33] and a “super nurse” culture[6] in which RNs perceive complete responsibility to care for assigned patients. Enhanced recognition of demand bursts during a work shift is a foundational step toward automated detection, early warning, and proactive intervention to level-load nurses' work across a shift.

Increased use of electronic work tools results in automatic production of large volumes of discrete time-series data[34] that afford increased observability of granular workplace activity. Exploration of operational data streams for detectible signs of strain is reasonable because earliest signs of work system compromise typically appear as small changes in facility operations.[19] The ambient workplace data used in this retrospective study was produced in real-time as a byproduct of patient care delivery. Feature definition was guided by nurses' qualitative understanding of the work system being studied,[20] which helps guard against the dangers of identifying spurious relationships in large datasets.[35] The current study demonstrated positive potential for future systems to continuously “listen” to operational data streams, potentially in combination with more traditional clinical data streams, to detect “hot spots” of activity on patient care units.

Future near-real-time analysis of ambient data streams may support dynamic recognition of changing workplace conditions and proactive intervention to increase reduce missed or delayed care. In addition, longitudinal analysis of the sources and effects of demand bursts, and development of proactive strategies to level-load work, may support organizational learning, consistent with the objectives of a learning health system.[36] As associations between patterns of clinician activity and overload are identified, improvement efforts can be launched to level-load work and eliminate unnecessary demand bursts.

A third key finding, which arose during the process of aggregating previously disparate data sources, was identification of the need for increased standardization of date–time formats and staff identifiers across information systems that automatically produce ambient data. Analogous to past standardization efforts applied to physiologic assessment data,[37] implementation of common identifiers for nurses, automated tagging of care events to specific work shifts, electronic capture of RN patient assignment, and normalization of date–time stamps across data sources are needed to support widespread use of ambient data in future quality-improvement initiatives. Standardization will substantially reduce the need for data preprocessing and will streamline future research that incorporates ambient workplace data.

Limitations and Recommendations for Future Study

Study findings are limited to a single patient care unit in an academic medical center which may limit generalizability to other sites. Ambient workplace data were originally generated for purposes other than detection of RN strain, and unplanned overtime is an imprecise estimate of RN strain. Verification of overtime status as planned or unplanned was not possible in this retrospective study. Additional research is needed to assess sociotechnical issues and ethical concerns related to automated workplace surveillance. Future studies may benefit from inclusion of EHR data as an additional data source, and collection of subjective assessments of RN strain as a refined study outcome. Feature extraction for smaller time frames and multiple levels of analysis may provide more granular insight and earlier recognition of emerging RN strain during a work shift. Future studies are also needed to assess the potential for features derived from ambient data, such as time spent using a communication device or percent of medications delivered via a syringe, to enhance EHR-based estimates of nursing workload.

Conclusion

Ambient data streams produced by care delivery tools can serve as information channels that contain signals related to unplanned overtime as a proxy indicator of RN strain as early as 8 hours into a work shift. This study represents an initial step toward a larger vision of early detection and proactive response to temporal demand bursts that present risk to patients and clinicians. Additional research is needed to develop machine learning models that incorporate additional data sources and operate on streaming data to facilitate real-time, collaborative human–machine monitoring of the hospital workplace for signs of RN strain.

Clinical Relevance Statement

Hospitals have a need for augmented recognition of periods of high production pressure that arise during nursing work shifts. Ambient data streams are valuable information channels that contain signals related to RN strain. Future operational decision support systems may benefit from inclusion of ambient workplace data streams, in addition to traditional input data sources such as the EHR. Earlier recognition of emerging RN strain has potential to reduce patient care delays and improve the RN work experience.

Multiple Choice Questions

1. Desirable characteristics of data used in dynamic workplace monitoring systems include automatic production and real-time capture as a byproduct of care delivery. All of the following data sources meet these criteria except:

   • Bedside nurse call records.

   • Electronic communication device records.

   • Medication dispensing system.

   • Narrative nursing documentation.

   Correct Answer: The correct answer is option d, narrative nursing documentation. Many care delivery tools including nurse call systems, electronic communication devices, and medication dispensing systems automatically produce transactional records and log file data as a result of human–computer interaction during care delivery. In contrast, production of narrative nursing documentation occurs after a nurse has assessed a patient, and requires intentional human effort. As a result, a delay may exist between the assessment event and the narrative documentation event, creating a delay in when narrative documentation is available for secondary analysis.

2. In this study, ambient workplace data are defined as:

   • Time-stamped transaction records and log file data automatically produced by non-electronic health record care delivery tools.

   • Electronic health record documentation produced by clinicians.

   • Time and motion study data collected by a human observer.

   • None of the above.

   Correct Answer: The correct answer is option a, time-stamped transaction records and log file data automatically produced by non-electronic health record care delivery tools. Ambient workplace data are defined in this study as time-stamped transaction records and log file data automatically produced by nonelectronic health record care delivery tools. Electronic health record documentation and collection of time and motion study data both require intentional human effort, rather than automated data production.

Conflict of Interest

None declared.

Acknowledgments

The authors would like to acknowledge nurse managers Adrienne McDougal and Mariah Hayes, members of the clinical expert panel, and information providers across multiple hospital departments who provided access to raw data from source systems.

Protection of Human and Animal Subjects

The study protocol was reviewed and approved by the Oregon Health and Science University Institutional Review Board.

• References

• 1 Oliveira AC, Garcia PC, Nogueira LS. Nursing workload and occurrence of adverse events in intensive care: a systematic review. Rev Esc Enferm USP 2016; 50 (04) 683-694
• 2 Bragadóttir H, Kalisch BJ, Tryggvadóttir GB. Correlates and predictors of missed nursing care in hospitals. J Clin Nurs 2017; 26 (11–12): 1524-1534
• 3 Aiken LH, Clarke SP, Sloane DM, Sochalski J, Silber JH. Hospital nurse staffing and patient mortality, nurse burnout, and job dissatisfaction. JAMA 2002; 288 (16) 1987-1993
• 4 Needleman J, Buerhaus P, Pankratz VS, Leibson CL, Stevens SR, Harris M. Nurse staffing and inpatient hospital mortality. N Engl J Med 2011; 364 (11) 1037-1045
• 5 Jones SK, Griep Y. “I can only work so hard before I burn out.” A time sensitive conceptual integration of ideological psychological contract breach, work effort, and burnout. Front Psychol 2018; 9: 131
• 6 Steege LM, Rainbow JG. Fatigue in hospital nurses - ‘Supernurse’ culture is a barrier to addressing problems: a qualitative interview study. Int J Nurs Stud 2017; 67: 20-28
• 7 See KC, Zhao MY, Nakataki E. , et al; SABA Study Investigators and the Asian Critical Care Clinical Trials Group. Professional burnout among physicians and nurses in Asian intensive care units: a multinational survey. Intensive Care Med 2018; 44 (12) 2079-2090
• 8 Poncet MC, Toullic P, Papazian L. , et al. Burnout syndrome in critical care nursing staff. Am J Respir Crit Care Med 2007; 175 (07) 698-704
• 9 Litvak E, Bisognano M. More patients, less payment: increasing hospital efficiency in the aftermath of health reform. Health Aff (Millwood) 2011; 30 (01) 76-80
• 10 Bodenheimer T, Sinsky C. From triple to quadruple aim: care of the patient requires care of the provider. Ann Fam Med 2014; 12 (06) 573-576
• 11 Woods D, Branlat M. Basic patterns in how adaptive systems fail. In: Hollnagel E, Paries J, Woods D, Wreathall J. , eds. Resilience Engineering in Practice: A Guidebook. Boca Raton, FL: Taylor & Francis Group; 2011: 127-144
• 12 Halpern SD. ICU capacity strain and the quality and allocation of critical care. Curr Opin Crit Care 2011; 17 (06) 648-657
• 13 Willis E, Harvey C, Thompson S, Pearson M, Meyer A. Work intensification and quality assurance: missed nursing care. J Nurs Care Qual 2018; 33 (02) E10-E16
• 14 Tucker AL, Spear SJ. Operational failures and interruptions in hospital nursing. Health Serv Res 2006; 41 (3, Pt 1): 643-662
• 15 Kalisch BJ, Xie B. Errors of omission: missed nursing care. West J Nurs Res 2014; 36 (07) 875-890
• 16 Roth C, Wieck KL, Fountain R, Haas BK. Hospital nurses' perceptions of human factors contributing to nursing errors. J Nurs Adm 2015; 45 (05) 263-269
• 17 Fasoli DR, Haddock KS. Results of an integrative review of patient classification systems. Annu Rev Nurs Res 2010; 28: 295-316
• 18 Hollnagel E. The ETTO Principle: Efficiency–Thoroughness Trade-Off: Why Things That Go Right Sometimes Go Wrong. Farnham: Ashgate Publishing Ltd; 2009
• 19 Chassin MR, Loeb JM. High-reliability health care: getting there from here. Milbank Q 2013; 91 (03) 459-490
• 20 Womack DM, Vuckovic NN, Steege LM, Eldredge DH, Hribar MR, Gorman PN. Subtle cues: qualitative elicitation of signs of capacity strain in the hospital workplace. Appl Ergon 2019; 81: 102893
• 21 Murphy K. Machine Learning: A Probabilistic Perspective. Cambridge, MA: Massachusetts Institute of Technology Press; 2012
• 22 Pedregosa P, Varoquaux G, Gramfort A. , et al. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research 2011; 12: 2825-2830
• 23 Myny D, Van Hecke A, De Bacquer D. , et al. Determining a set of measurable and relevant factors affecting nursing workload in the acute care hospital setting: a cross-sectional study. Int J Nurs Stud 2012; 49 (04) 427-436
• 24 Myny D, Van Goubergen D, Gobert M, Vanderwee K, Van Hecke A, Defloor T. Non-direct patient care factors influencing nursing workload: a review of the literature. J Adv Nurs 2011; 67 (10) 2109-2129
• 25 Carayon P, Gürses AP. A human factors engineering conceptual framework of nursing workload and patient safety in intensive care units. Intensive Crit Care Nurs 2005; 21 (05) 284-301
• 26 Kalisch BJ, Tschannen D, Lee H, Friese CR. Hospital variation in missed nursing care. Am J Med Qual 2011; 26 (04) 291-299
• 27 Patterson ES, Ebright PR, Saleem JJ. Investigating stacking: how do registered nurses prioritize their activities in real-time?. Int J Ind Ergon 2011; 41 (04) 389-393
• 28 Benner P, Tanner C, Chesla C. , eds. Expertise in Nursing Practice: Caring, Clinical Judgment, and Ethics. 2nd ed. New York: NY: Springer Publishing Company; 2009
• 29 Cook R, Rasmussen J. “Going solid”: a model of system dynamics and consequences for patient safety. Qual Saf Health Care 2005; 14 (02) 130-134
• 30 Tubbs-Cooley HL, Mara CA, Carle AC, Mark BA, Pickler RH. Association of nurse workload with missed nursing care in the neonatal intensive care unit. JAMA Pediatr 2019; 173 (01) 44-51
• 31 Ball JE, Murrells T, Rafferty AM, Morrow E, Griffiths P. ‘Care left undone’ during nursing shifts: associations with workload and perceived quality of care. BMJ Qual Saf 2014; 23 (02) 116-125
• 32 Steege LM, Pinekenstein BJ, Rainbow JG, Arsenault Knudsen É. Addressing occupational fatigue in nurses: current state of fatigue risk management in hospitals, part 1. J Nurs Adm 2017; 47 (09) 426-433
• 33 Padgett SM. Professional collegiality and peer monitoring among nursing staff: an ethnographic study. Int J Nurs Stud 2013; 50 (10) 1407-1415
• 34 Sharples S, Houghton RJ. The field becomes the laboratory? The impact of the contextual digital footprint on the discipline of E/HF. Ergonomics 2017; 60 (02) 270-283
• 35 Brennan PF, Bakken S. Nursing needs big data and big data needs nursing. J Nurs Scholarsh 2015; 47 (05) 477-484
• 36 Friedman CP, Wong AK, Blumenthal D. Achieving a nationwide learning health system. Sci Transl Med 2010; 2 (57) 57cm29
• 37 Matney SA, Settergren TT, Carrington JM, Richesson RL, Sheide A, Westra BL. Standardizing physiologic assessment data to enable big data analytics. West J Nurs Res 2017; 39 (01) 63-77

Publication History

Received: 21 May 2020

Accepted: 20 July 2020

Article published online:
16 September 2020