Accuracy of the Preferred Language Field in the Electronic Health Records of Two Canadian Hospitals

Abstract

Background The collection of race, ethnicity, and language (REaL) data from patients is advocated as a first step to identify, monitor, and improve health inequities. As a result, many health care institutions collect patients' preferred languages in their electronic health records (EHRs). These data may be used in clinical care, research, and quality improvement. However, the accuracy of EHR language data are rarely assessed.

Objectives This study aimed to audit the accuracy of EHR language data at two academic hospitals in Toronto, Ontario, Canada.

Methods The EHR language was compared with a patient's stated preferred language by interview. Language was dichotomized to English or non-English. Agreement between language documented in the EHR and patient-reported preferred language was calculated using sensitivity, specificity, and positive predictive value (PPV).

Results A total of 323 patients were interviewed, including 96 with a stated non-English preferred language. The sensitivity of the EHR for English-language preference was high at both hospitals: 100% at hospital A with a PPV of 88%, and 99% at hospital B with a PPV of 85%. However, the sensitivity of the EHR for non-English preference differed greatly between the two hospitals. The sensitivity was 81% with a PPV of 100% at hospital A and the sensitivity was 12% with a PPV of 60% at hospital B.

Conclusion The accuracy of the EHR for identifying non-English language preference differed greatly between the hospitals studied. Language data must be accurate for it to be used, and regular quality assurance is required.

Keywords

electronic health record - limited English proficiency - language barriers - equity - sociodemographic data

Background and Significance

Equity is a core component of health care quality, and yet it is often viewed as the “forgotten aim” of quality improvement.[1] [2] As a result, the collection of race, ethnicity, and language (REaL) data from patients is advocated as a first step to identify, monitor, and improve health inequities.[3] Without such data, inequities in care may be overlooked. In the United States, the Centers for Medicare and Medicaid Services mandates the collection of REaL data and recommends that these data be captured in electronic health records (EHRs).[4] Patients with limited English proficiency are a vulnerable population. They are at increased risk of adverse events when hospitalized and poor health outcomes when compared with English-proficient patients.[5] [6] [7] [8] Data on a patient's preferred language is therefore required to inform clinical care, such as the need for interpretation services, as well as to drive quality improvement and research.[3] [9] [10]

However, for language data to be useful, it must be accurate. Several studies have demonstrated issues in the accuracy of language data located in EHRs, including discrepancies between documented language preferences and patients' self-report.[11] [12] [13] Previous studies on the accuracy of EHR language data were conducted in the outpatient setting in jurisdictions with requirements to collect REaL data.[12] [13] Little is known about the accuracy of EHR language data for hospitalized patients in jurisdictions without mandates to collect such data in EHRs.

Our project aimed to systematically evaluate the quality of EHR language data at two hospitals in Toronto, Ontario, an ethnically diverse city in Canada with nearly 45% of residents reporting a mother tongue other than English or French, the official languages of the country.[14] The five most commonly spoken nonofficial languages in Toronto are Mandarin, Cantonese, Tagalog, Tamil, and Spanish.[14] Health care institutions in Ontario are not legally mandated to collect REaL data in their EHRs. Given that self-report is the most accurate method of collecting sociodemographic information,[15] we sought to audit the accuracy of a patients' preferred language (English or non-English) as recorded in the EHR, by comparing it to their stated preferred language by interview (English or non-English). We hypothesized that the EHR language would be moderately sensitive and highly specific for detecting a non-English preferred language at both hospitals based on previous work in this area.[11] [12] [13]

Methods

Participants

This prospective audit was conducted at two large urban academic hospitals in Toronto, Ontario, Canada. Patients admitted to the internal medicine wards of either hospital were eligible to participate. Data were collected over the course of several days between August 1, 2017 and November 22, 2017. A research assistant reviewed the census list for each internal medicine ward on the days of assessment. All patients were interviewed consecutively to identify their preferred language. Patients who were not available during the interview period were flagged for follow-up. If these patients could not be interviewed during a second attempt, they were excluded. Patients were also excluded if they were under airborne isolation precautions or were unable to communicate (e.g., severe cognitive impairment and aphasia).

Ethics

The Research Ethics Boards at both hospitals formally granted waivers for this project per local guidelines for quality improvement projects.

Data Collection

The research assistant asked patients a single question in English during the interview: what is your preferred language for health care communication? Interpretation services were not used.

At both hospitals, a patient's preferred language is recorded in the EHR by admitting clerks at the time of registration. At hospitals A and B, the default entry of the EHR language field is English. We dichotomized the preferred language listed in the EHR and the preferred language reported by participants to English or non-English to allow for a clear calculation of the accuracy of the EHR preferred language field. From a clinical perspective, a non-English preferred language in the EHR should prompt a health care professional to confirm a patient's preferred language and to assess the need for interpretation services.

Analysis

Following the interviews, the participants' responses were compared with the language preferences recorded in the EHR. We defined the overall accuracy of EHR language data as agreement with the participant's preferred language as determined by interview. We constructed 2 × 2 tables in Microsoft Excel and computed the sensitivity, specificity, and positive predictive value (PPV) of EHR data for identifying patients with English language preference and for identifying patients with non-English language preference. We calculated two-sided 95% confidence intervals for each measure using the Wilson score method with a continuity correction.[16]

Results

A total of 447 patients were screened for participation and 124 were excluded. Of the excluded patients, 32 (26%) were excluded because they were unable to communicate and 92 (74%) were excluded because they could not be reached on the second attempt ([Fig. 1]). There were no instances in which language data were missing from the EHR. Of the 323 patients who met inclusion criteria, 96 (29.7%) preferred a language other than English. Of the 96 patients who preferred a non-English language, 60 (62.5%) were correctly identified in the EHR language field. Of a total of 227 patients with a stated English language preference, 225 (99.1%) were correctly identified in the EHR. The overall sensitivity and specificity of the EHR language field for detecting a non-English language preference were 63 and 99%, respectively.

Fig. 1 Project flow diagram. EHR, electronic health record.

The sensitivity of the EHR language field for detecting a non-English language preference differed substantially between the two hospitals ([Table 1]). At hospital A, EHR data identified patients with non-English language preference with 81% sensitivity and 100% PPV, whereas at hospital B, EHR data identified patients with non-English language preference with 12% sensitivity and 60% PPV.

Table 1 Comparison of sensitivity, specificity, and PPV of the EHR for detecting non-English and English language preference

	Study (n = 323)	Hospital A (n = 163)	Hospital B (n = 160)
Non-English language preference (95% CI)
Sensitivity	0.63 (0.53–0.72)	0.81 (0.72–0.91)	0.12 (−0.01–0.24)
Specificity	0.99 (0.98–1.00)	1.00 (1.00–1.00)	0.99 (0.99–1.01)
PPV	0.97 (0.92–1.02)	1.00 (1.00–1.00)	0.60 (0.17–1.03)
English language preference (95% CI)
Sensitivity	0.99 (0.98–1.00)	1.00 (1.00–1.00)	0.99 (0.99–1.01)
Specificity	0.63 (0.53–0.72)	0.81 (0.72–0.91)	0.12 (−0.01–0.24)
PPV	0.86 (0.82–0.90)	0.88 (0.81–0.94)	0.85 (0.80–0.91)

Abbreviations: CI, confidence interval; EHR, electronic health record; PPV, positive predictive value.

Discussion

To our knowledge, this is the first multi-site inpatient audit of the accuracy of EHR preferred language data. We found that EHR language data were highly sensitive and moderately specific for identifying patients with a language preference for English. However, the accuracy of the EHR for detecting non-English language preference differed greatly between the two hospitals, which was inconsistent with our initial hypothesis. While the EHR of hospital A performed well, the sensitivity of the EHR at hospital B was lower than expected when compared with previous work. However, such work was conducted in settings where the collection of REaL data in EHRs is mandated, unlike in Ontario. For example, Azar et al compared patients' self-reported preferred languages to the values recorded in the EHR of a large health care organization in Northern California and found a mean concordance rate of 95%.[12] In a study conducted in Massachusetts, Klinger et al demonstrated modest performance of an EHR used in primary care in detecting non-English language preferences with a sensitivity of 79%.[13]

The absence of a legal mandate to record a patient's preferred language in the EHR in our jurisdiction may account for some of the differences seen between the two hospitals. Institutions may be more likely to develop processes to ensure the accurate collection of REaL data if they are required by law. In addition, the hospitals differed in their approach to language data collection in important ways. Admitting clerks at hospital A receive training on how to collect preferred language data, whereas clerks at hospital B do not. Training frontline staff how to ask about race, ethnicity, and language in a sensitive and effective manner is integral to collecting accurate data.[17] Clerks at hospital A are trained to ask all patients about their preferred language, and clerks at hospital B typically ask only those who appear to have difficulty communicating in spoken English. Self-reported language data, as collected at hospital A, are considered more accurate than observer-reported data.[15] Furthermore, when the default setting in the EHR is English, admitting clerks may be less likely to make changes if they have not received training on the importance of preferred language data collection.

Given the importance of accurate REaL data in identifying and improving health inequities, the Institute for Healthcare Improvement (IHI) recommends regular quality assurance.[18] It suggests validation sampling, comparing a patient's self-reported race, ethnicity, and language to the EHR in a random sample of patients. In addition, the number of unknown, other, or declined responses in the EHR should be tracked. The IHI also recommends that staff be observed to determine if REaL questions are posed in a manner consistent with the best practices and that patients also be observed to see how they respond.[4] As little as five observations can identify a lack of consistency in following institutional protocols.[18]

The findings of our audit support the IHI's call for quality assurance. Audits have been shown to improve the accuracy of language data in ambulatory care environments, but have not been examined in inpatient settings.[12] A recent study showed that targeted quality improvement interventions including (1) standardized training for staff, (2) EHR automatic alerts to prompt staff to enter REaL data for new patients, and (3) alerts to enter data for missing fields, increased the collection of REaL data from 71.7 to 84.1% at an academic health center serving a racially diverse patient population.[19] However, the accuracy of the data was not reported. Using quality improvement interventions to enhance the accuracy of REaL data remains an important priority for future work.

Other potential strategies to improve the accuracy of REaL data include the use of mobile applications on tablets or computers to facilitate self-report when patients register for outpatient appointments or in the emergency department. A recent study found that a sociodemographically diverse patient population could provide health histories using a web-based platform.[20] A systematic review found that survey responses collected via mobile applications were equivalent to responses collected using other platforms (paper, laptop, and personal digital assistant) and may improve data completeness.[21] While the use of information technology tools at the point of registration could minimize data entry errors and improve accuracy, hospitals must continue to perform audits to assess the accuracy of collected data. In addition, non-English speakers may require assistance from others to enter their data. In these instances, solutions could include displaying data entry forms in different languages or training admitting clerks to partner with interpreters to provide support.

When reporting of language data is mandatory, more accurate data may be collected. EHRs could then be used to study the impact of language on health outcomes and to inform interventions to improve quality of care and address health inequities. Such data could also be used to match patients to language concordant staff and to track if language needs are being met by an institution.[17] For example, through the collection of granular and accurate REaL data, the Palo Alto Medical Foundation created a culturally sensitive consult service to provide preventive cardiology care to South Asian patients, an identified high-risk group within their catchment.[12] Moreover, the use of professional interpretation is associated with improved quality of care, and accurate data on patients' language preferences may aid institutions in allocating resources for interpretation and other support services.[22]

Limitations

Our study had several limitations. First, we categorized languages into English and non-English. Although dichotomizing the language simplified our reporting of the accuracy of EHR data, it limited our ability to assess and comment on the quality of EHR data for identifying specific non-English languages. A non-English preferred language is also not a proxy for limited English proficiency as individuals who prefer a language other than English for their health care communication may still be proficient in reading, writing, or speaking English. Second, our results were limited to those patients who were able to be interviewed. As a result, 25%of patients we approached were excluded, potentially introducing selection bias. Third, we did not collect additional demographic data from patients such as gender or race/ethnicity. As a result, we are unable to compare the samples of the two institutions or examine for relationships between these demographic characteristics and language. This is an important area for future study. Fourth, we did not use interpreters to help ascertain the language preferences of participants. It is possible that some patients who reported an English language preference did not have a complete understanding of the question. Fifth, given that we found the EHRs of the two hospitals differed in their sensitivity for the detection of non-English language preference, our findings are unlikely to be generalizable to other institutions and local data audits are required.

Conclusion

We found important differences in the accuracy of language preference data in the EHR of two urban academic hospitals. The EHR data were highly sensitive for patients with English language preference at both hospitals, but the EHR of one hospital performed poorly in identifying patients with non-English language preference. The differences seen in data quality may be due in part to the absence of legal mandates for EHR REaL data collection in our jurisdiction and differences between the two hospitals in how they collect language data. However, for language data to be used in clinical care, research, and quality improvement, it must be accurate. Our findings highlight a need for standardizing the collection of data on preferred language and for regular quality assurance.

Clinical Relevance Statement

The collection of race, ethnicity, and language (REaL) data from patients is critical to identify, monitor, and improve health inequities. Organizations should develop strategies to collect and audit language data using existing informatics infrastructure.

Multiple Choice Questions

1. Which of the following is regarded as the “gold standard” for collecting preferred language data?

• Ensuring all staff receive cultural safety training.

• Inferring language from conversing with the patient.

• Asking the patient about their preferred language for health care discussions.

• Reviewing charts after patients have been discharged.

Correct Answer: The correct answer is option c. It is well established that self-report is the best way to collect data about patients' preferred language. The Health Research and Education Trust recommends asking “what language do you feel most comfortable speaking with your doctor or nurse?” Although it is important to ensure that frontline staff receive training in cultural safety, such training alone is not sufficient. Patients must be explicitly asked their language preference.

2. You are the new manager of Informatics and Quality Improvement (QI) at an academic hospital serving an ethnoracially and linguistically diverse population. Your first priority is to increase the collection of race, ethnicity, and language (REaL) data. What is the best QI intervention you could deploy?

• Standardized training for staff.

• EHR automatic alerts to prompt data entry.

• Alerts to enter data for missing fields.

• All of the above.

Correct Answer: The correct answer is option d. All of the above. Lee et al[19] demonstrated a greater than 10% absolute increase in the collection of REaL data after implementing all of the QI interventions above.

Conflict of Interest

A.A.V. reports that he is the Provincial Clinical Lead for Quality Improvement in General Internal Medicine with Health Quality Ontario.

Protection of Human and Animal Subjects

The study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects and received Research Ethics Board waivers from both institutional research boards as per local guidelines for quality improvement projects.

• References

• 1 Scott M, Rawal S. The promise of equity: a review of health equity research in high-impact quality improvement journals. Am J Med Qual 2018; 33 (03) 269-273
• 2 Wyatt R, Laderman M, Botwinick L, Mate K, Whittington J. Achieving Health Equity: A Guide for Health Care Organization. . Accessed August 7, 2020 at: https://www.esrdnetwork.org/sites/default/files/IHIAchievingHealthEquityWhitePaper.pdf
• 3 Chin MH. Using patient race, ethnicity, and language data to achieve health equity. J Gen Intern Med 2015; 30 (06) 703-705
• 4 Centers for Medicare and Medicaid Services. Inventory of Resources for Standardized Demographic and Language Data Collection. Accessed January 1, 2020 at: https://www.cms.gov/About-CMS/Agency-Information/OMH/Downloads/Data-Collection-Resources.pdf
• 5 John-Baptiste A, Naglie G, Tomlinson G. , et al. The effect of English language proficiency on length of stay and in-hospital mortality. J Gen Intern Med 2004; 19 (03) 221-228
• 6 Divi C, Koss RG, Schmaltz SP, Loeb JM. Language proficiency and adverse events in US hospitals: a pilot study. Int J Qual Health Care 2007; 19 (02) 60-67
• 7 Karliner LS, Kim SE, Meltzer DO, Auerbach AD. Influence of language barriers on outcomes of hospital care for general medicine inpatients. J Hosp Med 2010; 5 (05) 276-282
• 8 Rawal S, Srighanthan J, Vasantharoopan A, Hu H, Tomlinson G, Cheung AM. Association Between Limited English Proficiency and Revisits and Readmissions After Hospitalization for Patients With Acute and Chronic Conditions in Toronto, Ontario, Canada. JAMA 2019; 322 (16) 1605-1607
• 9 Wray R, Agic B, Bennett-AbuAyyash C. , et al. We ask because we care The Tri-Hospital + TPH Health Equity Data Collection Research Project Report. Accessed December, 2017 at: www.stmichaelshospital.com/quality/equity-data-collection-report.pdf
• 10 American Hospital Association. Reducing Health Care Disparities: Collection and Use of Race, Ethnicity and Language Data.. Accessed January 1, 2020 at: http://www.hpoe.org/resources/ahahret-guides/1431
• 11 McClure LA, Glaser SL, Shema SJ. , et al. Availability and accuracy of medical record information on language usage of cancer patients from a multi-ethnic population. J Immigr Minor Health 2010; 12 (04) 480-488
• 12 Azar KM, Moreno MR, Wong EC, Shin JJ, Soto C, Palaniappan LP. Accuracy of data entry of patient race/ethnicity/ancestry and preferred spoken language in an ambulatory care setting. Health Serv Res 2012; 47 (1 Pt 1): 228-240
• 13 Klinger EV, Carlini SV, Gonzalez I. , et al. Accuracy of race, ethnicity, and language preference in an electronic health record. J Gen Intern Med 2015; 30 (06) 719-723
• 14 2016 Census: Families, households and marital status; Language. Accessed June 10, 2019 at: https://www.toronto.ca/wp-content/uploads/2017/10/96e4-2016-Census-Backgrounder-Family-Households-Languages.pdf
• 15 Bierman AS, Lurie N, Collins KS, Eisenberg JM. Addressing racial and ethnic barriers to effective health care: the need for better data. Health Aff (Millwood) 2002; 21 (03) 91-102
• 16 Newcombe RG. Two-sided confidence intervals for the single proportion: comparison of seven methods. Stat Med 1998; 17 (08) 857-872
• 17 Institute of Medicine (US) Subcommittee on Standardized Collection of Race/Ethnicity Data for Healthcare Quality Improvement;. In: Ulmer C, McFadden B, Nerenz DR. , eds. Race, Ethnicity, and Language Data: Standardizationfor Health Care Quality Improvement. Washington, DC: The National Academies Press; 2009
• 18 Institute for Healthcare Improvement. Improving Health Equity: Guidance for Health Care Organizations. Accessed August 7, 2020 at: http://www.ihi.org/resources/Pages/Publications/Improving-Health-Equity-Guidance-for-Health-Care-Organizations.aspx
• 19 Lee WC, Veeranki SP, Serag H, Eschbach K, Smith KD. Improving the collection of race, ethnicity, and language data to reduce healthcare disparities: a case study from an academic medical center. Perspect Health Inf Manag 2016; 13 (Fall): 1g
• 20 Wu RR, Myers RA, Buchanan AH. , et al. Effect of sociodemographic factors on update of a patient-facing information technology family health history risk assessment platform. Appl Clin Inform 2019; 10 (02) 180-188
• 21 Marcano Belisario JS, Jamsek J, Huckvale K, O'Donoghue J, Morrison CP, Car J. Comparison of self-administered survey questionnaire responses collected using mobile apps versus other methods. Cochrane Database Syst Rev 2015; (07) MR000042
• 22 Karliner LS, Jacobs EA, Chen AH, Mutha S. Do professional interpreters improve clinical care for patients with limited English proficiency? A systematic review of the literature. Health Serv Res 2007; 42 (02) 727-754

Publication History

Received: 27 November 2019

Accepted: 24 July 2020

Article published online:
30 September 2020