Evaluating a Prototype Clinical Decision Support Tool for Chronic Pain Treatment in Primary Care

Abstract

Objectives The Chronic Pain Treatment Tracker (Tx Tracker) is a prototype decision support tool to aid primary care clinicians when caring for patients with chronic noncancer pain. This study evaluated clinicians' perceived utility of Tx Tracker in meeting information needs and identifying treatment options, and preferences for visual design.

Methods We conducted 12 semi-structured interviews with primary care clinicians from four health systems in Indiana. The interviews were conducted in two waves, with prototype and interview guide revisions after the first six interviews. The interviews included exploration of Tx Tracker using a think-aloud approach and a clinical scenario. Clinicians were presented with a patient scenario and asked to use Tx Tracker to make a treatment recommendation. Last, participants answered several evaluation questions. Detailed field notes were collected, coded, and thematically analyzed by four analysts.

Results We identified several themes: the need for clinicians to be presented with a comprehensive patient history, the usefulness of Tx Tracker in patient discussions about treatment planning, potential usefulness of Tx Tracker for patients with high uncertainty or risk, potential usefulness of Tx Tracker in aggregating scattered information, variability in expectations about workflows, skepticism about underlying electronic health record data quality, interest in using Tx Tracker to annotate or update information, interest in using Tx Tracker to translate information to clinical action, desire for interface with visual cues for risks, warnings, or treatment options, and desire for interactive functionality.

Conclusion Tools like Tx Tracker, by aggregating key information about past, current, and potential future treatments, may help clinicians collaborate with their patients in choosing the best pain treatments. Still, the use and usefulness of Tx Tracker likely relies on continued improvement of its functionality, accurate and complete underlying data, and tailored integration with varying workflows, care team roles, and user preferences.

Keywords

clinical decision support - pain - chronic - usability

Background and Significance

Over the past two decades, as electronic health record (EHR) adoption has increased, so has the volume of electronic patient data that clinicians can access at the point of care, and use to support treatment decisions.[1] [2] [3] [4] Computerized clinical decision support can facilitate user interaction with these data to visualize trends and prompt clinical action.[5] [6] [7] [8] Yet, the proliferation of EHRs and electronic clinical information has also led some to describe the information environment in primary care settings as “chaotic” and potentially harmful to clinician decision making and stress levels.[9] [10] [11] Accordingly, innovations are needed to improve clinicians' experiences and patient information management when using EHRs, including when caring for chronic health conditions, such as chronic noncancer pain.[12] [13] [14] [15] [16]

With nearly half of Americans suffering from one or more chronic conditions,[17] [18] primary care clinicians often make complex treatment decisions with limited time and information.[19] [20] [21] [22] [23] In chronic noncancer pain care, decision-making complexity and time constraints are particularly relevant.[24] Specifically, chronic noncancer pain typically has biopsychosocial roots and myriad treatment options.[25] [26] [27] Moreover, commonly used pain treatments, such as opioids and nonsteroidal anti-inflammatory drugs, are associated with numerous health risks and uncertain benefits.[25] [28] [29] [30] [31] Furthermore, patients with chronic noncancer pain often bring, to their clinical visits, long and varied histories of symptoms, treatments tried, and goals and preferences for their care. Because patients' histories are complex and individuals' values and preferences vary, chronic pain care may benefit from EHR-based clinical decision support[32] [33] [34] [35] that synthesizes and organizes key information, creating a structure that reduces time spent searching for and processing needed information. Moreover, this structure and time saving may allow primary care clinicians and their patients to engage more in shared decision making about pain treatment options.[36] [37] In chronic noncancer pain care, shared decision making typically involves clinicians and patients discussing pharmacologic treatments (including opioids), and nonpharmacologic treatments in the context of relative health risks, likely pain and functional benefits, as well as patients' goals and preferences.

The Chronic Pain Treatment Tracker (Tx Tracker) is a point-of-care visualization tool to better meet primary care clinicians' information needs, so they can engage patients in discussions about chronic noncancer pain treatments that best balance risks, benefits, and individual patient goals. Tx Tracker displays information about a patient's current and past treatments, potential new treatment options, trends in symptoms over time, and treatment risks, such as those related to opioid use disorder.[38] Prior to this study, we developed a Tx Tracker prototype based on a series of studies that identified primary care clinicians' information needs and decision-making challenges during visits by patients with chronic noncancer pain.[32] [35] [39] The prototype is an interactive application developed using the Axure prototyping tool (https://www.axure.com). While prior studies have evaluated decision support for opioid prescribing and chronic pain care,[40] [41] [42] this study is innovative because no prior evaluations focus on a tool to aid clinicians in choosing among many pain treatment options based on a presentation of past, current, and potential future treatments.

Objectives

In the context of point-of-care decision support for chronic noncancer pain, the objectives of this study were to (1) evaluate clinicians' perceived utility of Tx Tracker for identifying treatment options; (2) evaluate clinician's perceived clinical utility of Tx Tracker for meeting clinicians' point-of-care information needs; and (3) identify clinician preferences for layout and visual design of Tx Tracker. Our findings contribute to the ongoing development, implementation, and evaluation of Tx Tracker and other tools to support clinical information synthesis and treatment decision making for chronic pain and other chronic conditions.

Methods

Interviews

We conducted a scenario-based think-aloud evaluation study with 12 primary care clinicians from four health systems in the state of Indiana between February and May 2019. Scenario-based evaluations using think-aloud protocols are commonly used in technology usability studies, and in evaluations of complex medical decision making.[43] [44] [45] In support of an iterative design and evaluation process,[46] we conducted interviews in two waves. In the first wave, six interviews were conducted and preliminary review of participant feedback on usability was reviewed. Based on this preliminary analysis, the prototype interface was re-designed to address identified usability concerns, and additional interview questions were developed for consistency with the re-design. Next, we completed a second wave of six interviews. We recruited a total of 12 participants based on prior usability literature suggesting that most usability concerns can be identified with studies of 5 to 12 participants.[47] [48] Eligible participants were adult primary care clinicians (physicians) whose current practice included caring for patients with chronic noncancer pain. We recruited from four different health care systems that span rural and urban areas of Indiana. We recruited through our professional network, by approaching clinicians who had participated in a prior interview study about their information needs and decision-making processes for chronic noncancer pain.[39]

All 12 interviews consisted of background questions related to EHR utilization, current EHR functions that are useful, prototype exploration, the use of Tx Tracker in a hypothetical use case scenario, and evaluation questions. The moderator's guide, including the evaluation questions, use case scenario, and prototype exploration tasks, can be found in [Supplementary Appendix] (available in the online version). The background questions assessed clinicians' time spent using an EHR during work, number of terminals used in a typical workday, and tools used to track chronic pain treatment history in the EHR. These background questions were used to stimulate thought regarding the types of EHR tools clinicians use, and to place clinicians in a frame of mind that would aid recall of their daily clinical practice behavior. Next, clinicians freely explored the interactive prototype Tx Tracker on a notebook computer. Screenshots of the prototype are presented as [Figs. 1] and [2]. While exploring the prototype, clinicians were asked to think aloud to enable capture of their reactions, confusions, and interaction with the tool. Clinicians were encouraged to explore before proceeding to the use case scenario.

Fig. 1 Image of Chronic Pain Treatment Tracker prototype screen.

Fig. 2 Image of expanded medication section within Chronic Pain Treatment Tracker prototype.

Next, the use case scenario was presented as a case history of a fictional patient with chronic noncancer pain that the clinician had been treating for many years. The case history corresponded to patient-specific details presented in the Tx Tracker prototype. After the use case scenario was read aloud to the clinicians, they were asked to use the tool to review the patient's information, to make a treatment recommendation, and then answer evaluation questions. These evaluation questions assessed how Tx Tracker compared with interfaces and workflows in the clinicians' current EHRs, what clinicians perceived as most and least clinically useful in Tx Tracker, the types of patients Tx Tracker would be most useful, and when during routine care delivery, clinicians would expect to use Tx Tracker.

Nearly all testing interviews were conducted in person; however, two interviews were conducted remotely using an online video conferencing program. Ten interviews included both a detailed note taker (E.C.D.) and the main interviewer (S.M.D.), and two interviews utilized one person as note taker and the main interviewer (S.M.D.). Interviews lasted approximately 60 minutes. Utilizing Zoom audio and video recording, user audio and clinician interactions with the computer screen were recorded. Audio was not transcribed but was utilized to further enhance the interview notes, immediately following each interview. The interview notes were coded utilizing Dedoose. The study was approved by the Indiana University Institutional Review Board. Participants provided informed consent prior to participating the interviews. Each participant received a $100 gift card in recognition of their time spent participating.

Analysis

To enhance rigor via triangulation, four researchers (E.C.D., S.M.D., C.A.H., K.S.A.) engaged in different phases of the analysis. Three researchers (E.C.D., S.M.D., and C.A.H.) were involved in prior studies to develop Tx Tracker[38] while K.S.A. was not previously involved. Two researchers (E.C.D., S.M.D.) jointly reviewed their interview notes and created a single, detailed composite note for each section of the user testing experience. For interviews with only one researcher present, the second researcher reviewed the audio and video recording from the session to record notes and then completed the joint review process. All notes were recorded in an excel file, with rows and columns corresponding to participants and interview questions/sections, respectively.

A full coding analysis of the interviews was conducted by two researchers (E.C.D., S.M.D.). To begin, the reviewers utilized an inductive method, allowing the data to guide code development, and jointly coded the first two interviews from each of the waves (total of four) and utilized the identified codes to develop the initial codebook. By including interviews from both waves, we generated a more robust representative sample of user responses to the prototype, both before and after the design change. E.C.D. and S.M.D. then independently coded the remaining interviews, and conducted meetings to discuss and resolve any disagreements in coding.

After the initial coding was complete, another two researchers (K.S.A., C.A.H.) conducted the thematic analysis. To mitigate bias, K.S.A. was purposively brought in because she was not previously involved with the prototype development. To complete the thematic analysis, K.S.A. and C.A.H. first met with the initial coders (S.M.D., E.C.D.) to review the codes and collaboratively align them to the research objectives. They then conducted a thematic analysis to identify emergent themes related to each research objective. Specifically, the list of codes generated in the first review was assigned to research objectives based on applicability of the code to the question ([Table 1]). Using a process of upward extraction, each researcher independently reviewed the coded excerpts assigned to each objective, identified, and recorded themes found within. They then met to discuss and reach consensus on the themes based on their independent processes. To help mitigate the potential for bias associated with C.A.H.'s involvement in the earlier development of Tx Tracker, these consensus meetings involved identifying and discussing differences in the independent analyses. This process was repeated for each research objective over the course of 1 month (three consensus sessions in total). A near-final list of themes was developed based on a consensus between the two researchers (K.S.A. and C.A.H.). Lastly, to further triangulate across data products and confirm the identified themes, K.S.A. conducted a final review of each interview's audio and video.

Table 1 Mapping of qualitative codes to research questions

Research objective	Assigned codes
Evaluate the clinicians' perceived utility of Tx Tracker for identifying treatment options	• Comparison to current practice • Use in practice
Evaluate the perceived clinical utility of Tx Tracker for meeting clinicians' point-of-care information needs	Information: • Source • Organization • Missing • Useful
Identify clinician preferences for layout and visual design of Tx Tracker	• Desired features • Functionality

Results

Half of the participants where males (n = 6) and all have training in family medicine or internal medicine ([Table 2]). The participants practiced in 10 different primary care clinics that were part of four different health care systems, spanning rural and urban Indiana, and serving racially and socioeconomically diverse patient populations. In their clinics, participants used EHR systems from Epic, Cerner, or e-Clinical Works.

Table 2 Summary information about participating clinicians (N = 12)

Subject	Gender	Credentials	Specialty
1	M	MD, FAAFP	Family Medicine
2	F	MD	Family Medicine
3	F	MD	Internal Medicine
4	F	MD	Adult Medicine
5	M	MD	Primary Care
6	F	MD	Internal Medicine
7	M	MD	Family Medicine
8	M	MD	Family Medicine
9	M	MD, MS	Adult Medicine
10	M	MD, PhD	Internal Medicine
11	F	MD	Internal Medicine
12	F	MD	Family Medicine

[Table 3] provides illustrative quotes and summarizes the themes we identified for each research objective: (1) perceived clinical utility of Tx Tracker for identifying treatment options; (2) perceived clinical utility of Tx Tracker for meeting clinicians' point-of-care information needs; and (3) clinician preferences for layout and visual design of Tx Tracker.

Table 3 Identified themes and illustrative quotes

Research objectives and related themes	Illustrative quotes
Objective 1 themes: (a) Need for clinicians to be presented with a comprehensive patient history; (b) potential usefulness of Tx Tracker in patient discussions about treatment planning; (c) potential usefulness of Tx Tracker in treatment planning for patients with high uncertainty or risk.	4: “I think it would be nice to know the pain and function after that increase…if I knew that the person did much better – and they were able to work and to drive - then it may be worth continuing to increase. But if after an increase, they ask for another increase, but it didn't actually help, that would probably change what I did in the future.” 11: “The thing that is most useful is the history of what has happened in the past. I think patients get the most annoyed when I suggest something that we have done before and have had to stop. So, I think patients would like the most that I've remembered that we've tried that, and it didn't work.”
Objective 2 themes: (a) potential usefulness of Tx Tracker in aggregating scattered information; (b) variability in expectations about who and where in workflow Tx Tracker would be used; (c) skepticism about underlying EHR data quality, and information needed that was not in the prototype.	1: “This is one the hardest things in our current system. Even if I know somebody's gone to physical therapy, scrolling through, and finding those physical therapy appointments…honestly, I end up just totally relying on the patient for that and patients are either subconsciously or consciously notoriously inaccurate about reporting that.” 3: “Actually, it would be cool if this was something you could print and share with the patient because I think a lot of times, they aren't aware of what is all the history that has transpired and they might, some patients might benefit from being able to look at it.” 8: “One of the challenges is…who put the information there? For example…has chest pain but person put angina, but you cannot make that diagnosis…you need other information to understand the note…information has to be verified.”
Objective 3 themes: (a) Interest in using Tx Tracker to annotate or update information; (b) interest in using Tx Tracker to translate information to clinical action; (c) desire for interface with visual cues for risks, warnings, or treatment options; (d) desire for interactive functionality, such as search or information drill-down capability.	1: “As long as the condition for which the patient is having pain is identified…if in my assessment and plan, I have notes linked to that and I'd have orders linked to that and you could pull any of those things easily. The other thing is on any visit diagnosis, you can put comments on that visit diagnosis, so that would be a way to make it even easier. Just bring those comments in.” 2: “I don't know if these cautions just pop up due to reactions from medicines already tried or if the cautions will also pop up knowing what their current medical problems are. It would be nice to have both because if a patient has a past history of GI bleed or if they have chronic kidney disease it would be nice for that to just pop up without me knowing to go search for it.” 11: “It would be nice to put a note like ‘plan to increase to 120 tables for one month then go back 90’.”

Abbreviations: EHR, electronic health record; GI, gastrointestinal.

Objective 1

Perceived clinical utility of Tx Tracker for Identifying Treatment Options

We identified three themes during the thematic analysis related to clinical utility of Tx Tracker. The first was a strong need for clinicians to be presented with a comprehensive patient history. Clinicians requested detailed treatment histories, preferably with context describing the rationale behind starting and stopping certain treatment options. Desired patient history included medication dosage/duration, dates of visits to all relevant providers, referrals, missed visits, and indications as to treatment adherence. Contextual information included specific pain diagnosis(es), goals, comorbidities, patient preferences, pain and function outcomes, and relevant laboratories (e.g., creatinine).

Second, clinicians reported that Tx Tracker would be useful to support discussions with patients about treatment planning. Clinicians indicated that patients were dissatisfied during visits when their provider did not know their full medical history, including what treatments had been tried in the past in relation to a list of possible treatment options. Clinicians also reported value in the ability to generate patient-specific printable informational handouts via Tx Tracker.

Third, clinicians indicated that a tool like Tx Tracker would be helpful in treatment planning for patients with high uncertainty or risk. Specifically, participants mentioned potential usefulness when planning treatment for new patients, colleagues' patients with whom they were unfamiliar, patients with uncontrolled pain, or patients who were at higher risk of adverse outcomes such as opioid misuse or overdose.

Objective 2

Perceived Clinical Utility of Tx Tracker for Meeting Clinicians' Point-of-Care Information Needs

Overall, clinicians found Tx Tracker to be useful in aggregating critical information that, in their current practice, was often scattered throughout the EHR and took time to find and synthesize into a coherent whole. Participants consistently noted that this would be applicable to other complex, chronic conditions as well. However, clinicians indicated that categories or groupings of clinical concepts (e.g., medications, referrals) should be well defined and understandable by a Tx Tracker user. Additionally, clinicians said all the treatment options (e.g., nutrition as well as medication) should be represented in Tx Tracker.

Clinicians varied in their expectations for the right place in the workflow and right team member to be utilizing Tx Tracker. More clinicians indicated that it would be useful for previsit patient review, though several clinicians also indicated in-visit usefulness. However, some clinicians expressed skepticism about having adequate time to use Tx Tracker, given time-constrained work schedules. They indicated that nursing or other team members may need to be responsible for data input and review. Also, some clinicians indicated that Tx Tracker would support other downstream workflows, such as prior authorization and documentation for billing and regulatory compliance.

Importantly, clinicians expressed skepticism about Tx Tracker reliably containing accurate, trustworthy, and relevant information. Clinicians expressed concerns about underlying EHR data quality and, thus, the ability of Tx Tracker to consistently display the most relevant information. Clinicians also noted several information needs that were not met by the current prototype. Examples of these information needs included the association of comorbidities to cautions/warnings, pain and function levels, laboratory history and trends, health information exchange integration of outside records, nonclinical treatment modalities, and insurance coverage. Participants also expressed value in Tx Tracker explicitly indicating when critical information (e.g., important laboratory values) was missing from the patient's EHR.

Objective 3

Clinician Preferences for Layout and Visual Design of Tx Tracker

First, participating clinicians expressed interest in using Tx Tracker to annotate or update information, which was not part of the current prototype. Annotating or updating includes items such as summarizing discussions, commenting on a note from a specialist, adding free text descriptions on past treatments, and the ability to drag and drop treatments across Tx Tracker sections.

Second, clinicians expressed a need for Tx Tracker to translate information into clinical action, functionality which was largely not present in the prototype. For example, interviewees noted the value of actions such as writing orders or assessing risk by checking a prescription drug monitoring program (PDMP) database. Also, participants suggested the value of translating information in Tx Tracker to actions such as messaging fellow clinicians, writing notes, and educating patients. Several participants mentioned the added utility of being able to print out patient education materials.

Third, through visual cues, clinicians indicated a preference for Tx Tracker calling attention to important risks or warnings, such as those related to pill counts, prescriptions, or comorbidities. Participants suggested additional use of colors and prominence of screen location to indicate the level of caution being highlighted. Participants also suggested using visual cues to relate treatment options to specific types of pain for which the option may be effective.

Finally, clinicians expressed that Tx Tracker should allow dynamic information seeking based on their use preferences, such as interactivity options that allow them to condense or expand on details about less or more critical information. Participants also suggested search functionality, such as to help them find critical laboratory values.

Discussion

To our knowledge, this study is the first to evaluate a decision support tool to aid clinicians in choosing among pain treatment options by summarizing past, current, and potential future treatments. With this summary, clinicians may be able to compare treatment options more efficiently, and better engage patients in discussing and choosing treatments that best balance risks, benefits, and patient goals (i.e., shared decision making). Indeed, clinicians in our study indicated the potential value of using Tx Tracker to facilitate discussion of treatment options based on each patient's unique medical history, to compare treatments to outcomes by visualizing pain versus treatments used over time, and to share patient education materials. An imperative in clinical decision support is providing the right information to the right person in the right format.[4] [33] By providing clinicians with summaries of past, current, and potential future treatments as well as with outcomes information and education materials, Tx Tracker may make it easier for clinicians and patients to actively discuss and choose the best pain treatments. Notably, participants expressed an interest in using Tx Tracker for patients who are at higher risk, such as for opioid misuse or overdose. When opioids are involved, shared decision making may be uniquely challenging.[49] Thus, Tx Tracker should continue to be evaluated in more realistic scenarios involving patient and clinician communication about opioid use for chronic pain.

This study identified other important findings related to Tx Tracker's potential to routinely meet primary care clinicians' information needs. Participants expressed support for the utility of an EHR-based tool that aggregates and synthesizes information that is often scattered across multiple places in an EHR. At the same time, clinicians expressed skepticism about the accuracy and relevancy of underlying EHR data on which the Tx Tracker display relies. Participating clinicians also expressed interest in including additional information in Tx Tracker, such as records from other providers, insurance coverage information, and laboratory results. Without complete and accurate information in the EHR, such as about past treatments tried and pain outcomes, Tx Tracker's usability in practice may be limited. This finding offers important guidance on future design and implementation work. Future design work should more closely assess the quality of real-world EHR data for patients with chronic pain, and its usability when presented via Tx Tracker or similar tools. This finding also suggests the value of continued work to improve EHR data quality, such as through health information exchange,[50] [51] standards-based data collection,[52] [53] and reliable natural language processing methods for extracting symptom and treatment information from unstructured clinical notes.[54] [55] Moreover, as we continue to develop Tx Tracker, we will consider ways to effectively include other pain-relevant data types, such as imaging and nonprimary care specialist notes and reports.

This study's finding also reinforced that the value of Tx Tracker will only be realized if it is well integrated with clinical workflows. Consistent with previous findings,[14] [56] clinicians varied in their expectations for when in clinical workflows, and by what types of users, Tx Tracker would be most useful. Some clinicians indicated a preference for previsit use, while others indicated a preference for in-visit use. These differences could reflect variability across multiple factors, such as clinician work style, patient visit volumes, support staff roles, or technology layouts within clinics (e.g., computer, monitor, and peripherals).

Relatedly, participating clinicians varied in their views of whom should use Tx Tracker. Some suggesting that medical assistants or nursing staff would be needed to input or review data. Some skepticisms were about having enough time to fully utilize Tx Tracker. These findings are consistent with prior clinical decision support research that finds that a strong understanding of clinical workflow is key to successful implementation.[56] [57] [58] [59] Furthermore, these findings reinforce the need for continued focus on assessing variability in clinician preferences and workflows, and for using socio-technical design and implementation strategies to tailor decision support for adoption and use across diverse clinical settings.[56] [57] [59] [60] In future work, we will also explore Tx Tracker's potential use by other specialties, such as by pain specialist physicians or by multidisciplinary teams conducting pain rounds or pain clinics.

This study offered several specific recommendations for improving the design of Tx Tracker. Clinicians suggested functionality to facilitate clinical action, such as note writing, order writing, messaging other providers, and accessing outside information (e.g., PDMP database). Clinical decision support that is actionable and helps clinicians more efficiently complete required tasks is more likely to be adopted and maintained in clinical practice.[61] [62] Finally, this study reinforced the potential for improving Tx Tracker design through better use of visual cues along with interactive features for high-level overview of information or examining the specific details.[63] In the context of chronic pain, such features may be useful for avoiding high-risk treatment choices, such as prescribing an opioid to patients with a history of opioid misuse or substance use disorder.

This study is not without limitations. First, this was a qualitative research study with a modest number of participants who practice adult primary care medicine in one state. These findings may not transfer beyond the sample, and we cannot provide quantitative estimates of clinician perceptions. Relatedly, while studies of 5 to 12 participants are thought to identify most usability concerns,[47] [48] such rules of thumb are imperfect. Thus, our study likely did not capture all possible user concerns. Still, the sample was diverse in terms of participant gender, patients served by participants, and health care organization. Second, the study was focused on a single clinical decision support tool designed to support chronic pain treatment decisions in primary care settings. While participants noted the potential use of a Tx Tracker-like tool for other conditions, our findings here may not generalize to decision support for other conditions. Third, given the preliminary nature of our study, we were not able to access actual use of Tx Tracker in clinical practice. We are currently conducting a randomized controlled trial of a decision support tool which includes Tx Tracker. This integration of Tx Tracker in the Epic EHR is ongoing at two academic health centers, and its evaluation as part of a pragmatic randomized controlled trial (ClinicalTrials.gov Registration number NCT04295135).

Conclusion

The widespread use of EHRs and proliferation of electronic patient information necessitate research on how to best organize information and communicate with clinicians at the point of care, especially when they are managing patients with complex and costly chronic conditions. Tools like Tx Tracker, by aggregating key information about past, current, and potential future treatments, may help clinicians collaborate with their patients in choosing the best pain treatments. Still, the use and usefulness of Tx Tracker likely relies on continued improvement of its functionality, accurate and complete underlying data, and tailored integration with varying workflows, care team roles, and user preferences.

Clinical Relevance Statement

Managing complex conditions, such as chronic noncancer pain, requires significant time and information, which are often not readily available to primary care clinicians. Decision support for this costly and prevalent condition has the potential to ease clinician cognitive burden, facilitate patient communication, and promote better patient health outcomes.

Multiple Choice Questions

1. When considering clinical decision support for management of chronic pain, what elements did clinicians identify as important?

   • Comprehensive patient history and dynamic interaction.

   • Only the current medications and active treatments be listed.

   • Visual cues to draw attention to warnings and potential treatment interactions.

   • Both a and c are important for usefulness and usability.

   Correct Answer: The correct answer is option d. Clinicians expressed the importance of having a comprehensive patient history to guide treatment decisions as well as indicating patients feel strongly that clinicians should be aware of their history. Additionally, clinicians have a desire to act upon the information being presented to them, in terms of both recommendations and warnings related to treatment options. The ability to annotate and/or document directly within the system is also preferred (Objectives 1 and 3).

2. Among clinicians, there are still varied opinions on what implementation aspect of clinical decision support?

   • The need for actionable information.

   • When in the workflow it should be utilized.

   • Whether information presented should be comprehensive.

   • There are no varied opinions in this space.

   Correct Answer: The correct answer is option b. A consistent theme throughout the interviews is the uncertainty as to when these types of clinical decision support are most helpful. There are some who expressed the desire to review this with the patient during the appointment and others who suggested the review may need to take place prior to the appointment and perhaps by nursing staff. There are concerns related to having enough time available to utilize the tool and as to whether the underlying EHR data will support accuracy (Objective 2, paragraph 2).

Conflict of Interest

None declared.

Note

The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality.

Protection of Human and Animal Subjects

This study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects and was approved by the Indiana University Institutional Review Board.

• References

• 1 Middleton B, Sittig DF, Wright A. Clinical decision support: a 25 year retrospective and a 25 year vision. Yearb Med Inform 2016; 25 (Suppl. 01) S103-S116
• 2 Nanji KC, Garabedian PM, Shaikh SD. et al. Development of a perioperative medication-related clinical decision support tool to prevent medication errors: an analysis of user feedback. Appl Clin Inform 2021; 12 (05) 984-995
• 3 Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med 2020; 3 (01) 17
• 4 Osheroff JA, Teich J, Levick D. et al. Improving Outcomes with Clinical Decision Support: An Implementer's Guide. 2nd ed. Boca Raton, FL: CRC Press; 2012
• 5 Shortliffe EH, Sepúlveda MJ. Clinical decision support in the era of artificial intelligence. JAMA 2018; 320 (21) 2199-2200
• 6 Van Dort BA, Zheng WY, Sundar V, Baysari MT. Optimizing clinical decision support alerts in electronic medical records: a systematic review of reported strategies adopted by hospitals. J Am Med Inform Assoc 2021; 28 (01) 177-183
• 7 Unberath P, Prokosch HU, Kraus S. A REST service for the visualization of clinical time series data in the context of clinical decision support. Stud Health Technol Inform 2019; 258: 26-30
• 8 Berner ES, La Lande TJ. Overview of clinical decision support systems. In: Berner ES. ed. Clinical Decision Support Systems. New York, NY: Springer; 2007: 3-22
• 9 Beasley JW, Wetterneck TB, Temte J. et al. Information chaos in primary care: implications for physician performance and patient safety. J Am Board Fam Med 2011; 24 (06) 745-751
• 10 Schaefer E, Werner N, Scanlon M. Information chaos in the electronic health record as a threat to patient safety. In: Stephanidis C, Antona M, Ntoa S. eds. HCI International 2021 - Late Breaking Posters. Vol 1499. Communications in Computer and Information Science. New York, NY: Springer International Publishing; 2021: 189-196
• 11 Kroth PJ, Morioka-Douglas N, Veres S. et al. Association of electronic health record design and use factors with clinician stress and burnout. JAMA Netw Open 2019; 2 (08) e199609
• 12 Harle CA, Marlow NM, Schmidt SOF. et al. The effect of EHR-integrated patient-reported outcomes on satisfaction with chronic pain care. Am J Manag Care 2016; 22 (12) e403-e408
• 13 O'Connor PJ, Sperl-Hillen JM, Rush WA. et al. Impact of electronic health record clinical decision support on diabetes care: a randomized trial. Ann Fam Med 2011; 9 (01) 12-21
• 14 Bhat S, Derington CG, Trinkley KE. Clinicians' values and preferences for medication adherence and cost clinical decision support in primary care: a qualitative study. Appl Clin Inform 2020; 11 (03) 405-414
• 15 Dorr DA, D'Autremont C, Pizzimenti C. et al. Assessing data adequacy for high blood pressure clinical decision support: a quantitative analysis. Appl Clin Inform 2021; 12 (04) 710-720
• 16 Brunner MC, Sheehan SE, Yanke EM. et al. Joint design with providers of clinical decision support for value-based advanced shoulder imaging. Appl Clin Inform 2020; 11 (01) 142-152
• 17 Fried LP. America's health and health care depend on preventing chronic disease. Huffington Post. Published March 8, 2017. Accessed March 20, 2020 at: https://www.huffpost.com/entry/americas-health-and-healthcare-depends-on-preventing_b_58c0649de4b070e55af9eade
• 18 Chronic Care. A call to action for health reform. Accessed March 20, 2020 at: https://assets.aarp.org/rgcenter/health/beyond_50_hcr_conditions.pdf
• 19 Østbye T, Yarnall KS, Krause KM, Pollak KI, Gradison M, Michener JL. Is there time for management of patients with chronic diseases in primary care?. Ann Fam Med 2005; 3 (03) 209-214
• 20 Tai-Seale M, Bolin J, Bao X, Street R. Management of chronic pain among older patients: inside primary care in the US. Eur J Pain 2011; 15 (10) 1087.e1-1087.e8
• 21 Tsiga E, Panagopoulou E, Sevdalis N, Montgomery A, Benos A. The influence of time pressure on adherence to guidelines in primary care: an experimental study. BMJ Open 2013; 3 (04) e002700
• 22 Freedman S, Golberstein E, Huang TY, Satin DJ, Smith LB. Docs with their eyes on the clock? The effect of time pressures on primary care productivity. J Health Econ 2021; 77: 102442
• 23 Oakes AH, Adusumalli S, Snider CK, Rareshide CAL, Patel MS. Variation in cardiologist statin prescribing by clinic appointment time. J Am Coll Cardiol 2021; 77 (05) 661-662
• 24 Neprash HT, Barnett ML. Association of primary care clinic appointment time with opioid prescribing. JAMA Netw Open 2019; 2 (08) e1910373
• 25 McDonagh MS, Selph SS, Buckley DI. et al. Nonopioid Pharmacologic Treatments for Chronic Pain. Rockville, MD: Agency for Healthcare Research and Quality (US);. 2020 . Accessed February 23, 2022 at: http://www.ncbi.nlm.nih.gov/books/NBK556277/
• 26 Miaskowski C, Blyth F, Nicosia F. et al. A biopsychosocial model of chronic pain for older adults. Pain Med 2020; 21 (09) 1793-1805
• 27 Gatchel RJ, Peng YB, Peters ML, Fuchs PN, Turk DC. The biopsychosocial approach to chronic pain: scientific advances and future directions. Psychol Bull 2007; 133 (04) 581-624
• 28 Bendtsen P, Hensing G, Ebeling C, Schedin A. What are the qualities of dilemmas experienced when prescribing opioids in general practice?. Pain 1999; 82 (01) 89-96
• 29 Bergman AA, Matthias MS, Coffing JM, Krebs EE. Contrasting tensions between patients and PCPs in chronic pain management: a qualitative study. Pain Med 2013; 14 (11) 1689-1697
• 30 Dobscha SK, Corson K, Perrin NA. et al. Collaborative care for chronic pain in primary care: a cluster randomized trial. JAMA 2009; 301 (12) 1242-1252
• 31 Ho KY, Gwee KA, Cheng YK, Yoon KH, Hee HT, Omar AR. Nonsteroidal anti-inflammatory drugs in chronic pain: implications of new data for clinical practice. J Pain Res 2018; 11: 1937-1948
• 32 Harle CA, Apathy NC, Cook RL. et al. Information needs and requirements for decision support in primary care: an analysis of chronic pain care. AMIA Annu Symp Proc 2018; 2018: 527-534
• 33 Marcial LH, Richardson JE, Lasater B. et al. The imperative for patient-centered clinical decision support. EGEMS (Wash DC) 2018; 6 (01) 12
• 34 Marcial LH, Blumenfeld B, Harle C. et al. Barriers, facilitators, and potential solutions to advancing interoperable clinical decision support: multi-stakeholder consensus recommendations for the opioid use case. AMIA Annu Symp Proc 2020; 2019: 637-646
• 35 Diiulio J, Militello LG, Andraka-Christou BT. et al. Factors that influence changes to existing chronic pain management plans. J Am Board Fam Med 2020; 33 (01) 42-50
• 36 O'Brien EM, Staud RM, Hassinger AD. et al. Patient-centered perspective on treatment outcomes in chronic pain. Pain Med 2010; 11 (01) 6-15
• 37 Becker KL. Tell me your dreams and goals: structuring communication exchanges to improve patient-centered care with chronic pain patients. Appl Nurs Res 2020; 53: 151248
• 38 Harle CA, DiIulio J, Downs SM. et al. Decision-centered design of patient information visualizations to support chronic pain care. Appl Clin Inform 2019; 10 (04) 719-728
• 39 Militello LG, Hurley RW, Cook RL. et al. Primary care clinicians' beliefs and strategies for managing chronic pain in an era of a national opioid epidemic. J Gen Intern Med 2020; 35 (12) 3542-3548
• 40 Trafton JA, Martins SB, Michel MC. et al. Designing an automated clinical decision support system to match clinical practice guidelines for opioid therapy for chronic pain. Implement Sci 2010; 5 (01) 26
• 41 Meadows G, Moesel C, Sebastian S, Winters D. The MITRE Corporation. Factors to Consider in Managing Chronic Pain: A Pain Management Summary. 2018 . Accessed February 23, 2021 at: https://digital.ahrq.gov/sites/default/files/docs/citation/cds-artifact-enhancement.pdf
• 42 Harle CA, Listhaus A, Covarrubias CM. et al. Overcoming barriers to implementing patient-reported outcomes in an electronic health record: a case report. J Am Med Inform Assoc 2016; 23 (01) 74-79
• 43 Belden JL, Grayson R, Barnes J. Defining and testing EMR usability: principles and proposed methods of EMR usability evaluation and rating. . Published 2009 at: http://hdl.handle.net/10355/3719.
• 44 Ericsson KA, Simon HA. Verbal reports as data. Psychol Rev 1980; 87 (03) 215-251
• 45 Kushniruk AW. Analysis of complex decision-making processes in health care: cognitive approaches to health informatics. J Biomed Inform 2001; 34 (05) 365-376
• 46 Kushniruk A. Evaluation in the design of health information systems: application of approaches emerging from usability engineering. Comput Biol Med 2002; 32 (03) 141-149
• 47 Nielsen J. Why you only need to test with 5 users. Published online March 18, 2000. Accessed February 23, 2022 at: https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/
• 48 Hwang W, Salvendy G. Number of people required for usability evaluation: the 10 ±  2 rule. Commun ACM 2010; 53 (05) 130-133
• 49 Matthias MS, Talib TL, Huffman MA. Managing chronic pain in an opioid crisis: what is the role of shared decision-making?. Health Commun 2020; 35 (10) 1239-1247
• 50 Vest JR, Ancker JS. Health information exchange in the wild: the association between organizational capability and perceived utility of clinical event notifications in ambulatory and community care. J Am Med Inform Assoc 2017; 24 (01) 39-46
• 51 D'Amore J, Bouhaddou O, Mitchell S. et al. Interoperability progress and remaining data quality barriers of certified health information technologies. AMIA Annu Symp Proc 2018; 2018: 358-367
• 52 van der Bij S, Khan N, Ten Veen P, de Bakker DH, Verheij RA. Improving the quality of EHR recording in primary care: a data quality feedback tool. J Am Med Inform Assoc 2017; 24 (01) 81-87
• 53 Sayeed R, Gottlieb D, Mandl KD. SMART Markers: collecting patient-generated health data as a standardized property of health information technology. NPJ Digit Med 2020; 3 (01) 9
• 54 Kharrazi H, Anzaldi LJ, Hernandez L. et al. The value of unstructured electronic health record data in geriatric syndrome case identification. J Am Geriatr Soc 2018; 66 (08) 1499-1507
• 55 Naseri H, Kafi K, Skamene S. et al. Development of a generalizable natural language processing pipeline to extract physician-reported pain from clinical reports: Generated using publicly-available datasets and tested on institutional clinical reports for cancer patients with bone metastases. J Biomed Inform 2021; 120: 103864
• 56 Trinkley KE, Blakeslee WW, Matlock DD. et al. Clinician preferences for computerised clinical decision support for medications in primary care: a focus group study. BMJ Health Care Inform 2019;26(01):0
• 57 Peute LW, Aarts J, Bakker PJM, Jaspers MWM. Anatomy of a failure: a sociotechnical evaluation of a laboratory physician order entry system implementation. Int J Med Inform 2010; 79 (04) e58-e70
• 58 Maviglia SM, Zielstorff RD, Paterno M, Teich JM, Bates DW, Kuperman GJ. Automating complex guidelines for chronic disease: lessons learned. J Am Med Inform Assoc 2003; 10 (02) 154-165
• 59 Greenes RA, Bates DW, Kawamoto K, Middleton B, Osheroff J, Shahar Y. Clinical decision support models and frameworks: Seeking to address research issues underlying implementation successes and failures. J Biomed Inform 2018; 78: 134-143
• 60 Berg M, Aarts J, van der Lei J. ICT in health care: sociotechnical approaches. Methods Inf Med 2003; 42 (04) 297-301
• 61 Miller K, Mosby D, Capan M. et al. Interface, information, interaction: a narrative review of design and functional requirements for clinical decision support. J Am Med Inform Assoc 2018; 25 (05) 585-592
• 62 Kannry J, McCullagh L, Kushniruk A, Mann D, Edonyabo D, McGinn T. A framework for usable and effective clinical decision support: experience from the iCPR randomized clinical trial. EGEMS (Wash DC) 2015; 3 (02) 1150
• 63 Shneiderman B. The eyes have it: a task by data type taxonomy for information visualizations. In: Bederson BB, Shneiderman B. eds. The Craft of Information Visualization, Interactive Technologies. San Francisco, CA: Morgan Kaufmann; 2003: 364-371

Publication History

Received: 02 November 2021

Accepted: 08 April 2022

Article published online:
01 June 2022