Appl Clin Inform 2022; 13(03): 602-611
DOI: 10.1055/s-0042-1749332
Research Article

Evaluating a Prototype Clinical Decision Support Tool for Chronic Pain Treatment in Primary Care

Katie S. Allen
1   Health Policy and Management, Richard M. Fairbanks School of Public Health, IUPUI, Indianapolis, Indiana, United States
2   Center for Biomedical Informatics, Regenstrief Institute, Inc., Indianapolis, Indiana, United States
Elizabeth C. Danielson
3   Center for Education in Health Sciences, Northwestern University Feinberg School of Medicine, Chicago, Illinois, United States
Sarah M. Downs
4   Division of Internal Medicine, Indiana University School of Medicine, Indianapolis, Indiana, United States
Olena Mazurenko
1   Health Policy and Management, Richard M. Fairbanks School of Public Health, IUPUI, Indianapolis, Indiana, United States
Julie Diiulio
5   Health Outcomes and Biomedical Informatics, Applied Decision Science, LLC, Dayton, Ohio, United States
Ramzi G. Salloum
6   University of Florida, Gainesville, Florida, United States
Burke W. Mamlin
2   Center for Biomedical Informatics, Regenstrief Institute, Inc., Indianapolis, Indiana, United States
4   Division of Internal Medicine, Indiana University School of Medicine, Indianapolis, Indiana, United States
Christopher A. Harle
2   Center for Biomedical Informatics, Regenstrief Institute, Inc., Indianapolis, Indiana, United States
6   University of Florida, Gainesville, Florida, United States
› Author Affiliations
Funding This project was supported by Grant R01HS023306 from the Agency for Healthcare Research and Quality, Designing User-Centered Decision Support Tools for Primary Care Pain Management.


Objectives The Chronic Pain Treatment Tracker (Tx Tracker) is a prototype decision support tool to aid primary care clinicians when caring for patients with chronic noncancer pain. This study evaluated clinicians' perceived utility of Tx Tracker in meeting information needs and identifying treatment options, and preferences for visual design.

Methods We conducted 12 semi-structured interviews with primary care clinicians from four health systems in Indiana. The interviews were conducted in two waves, with prototype and interview guide revisions after the first six interviews. The interviews included exploration of Tx Tracker using a think-aloud approach and a clinical scenario. Clinicians were presented with a patient scenario and asked to use Tx Tracker to make a treatment recommendation. Last, participants answered several evaluation questions. Detailed field notes were collected, coded, and thematically analyzed by four analysts.

Results We identified several themes: the need for clinicians to be presented with a comprehensive patient history, the usefulness of Tx Tracker in patient discussions about treatment planning, potential usefulness of Tx Tracker for patients with high uncertainty or risk, potential usefulness of Tx Tracker in aggregating scattered information, variability in expectations about workflows, skepticism about underlying electronic health record data quality, interest in using Tx Tracker to annotate or update information, interest in using Tx Tracker to translate information to clinical action, desire for interface with visual cues for risks, warnings, or treatment options, and desire for interactive functionality.

Conclusion Tools like Tx Tracker, by aggregating key information about past, current, and potential future treatments, may help clinicians collaborate with their patients in choosing the best pain treatments. Still, the use and usefulness of Tx Tracker likely relies on continued improvement of its functionality, accurate and complete underlying data, and tailored integration with varying workflows, care team roles, and user preferences.


The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality.

Protection of Human and Animal Subjects

This study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects and was approved by the Indiana University Institutional Review Board.

Supplementary Material

Publication History

Received: 02 November 2021

Accepted: 08 April 2022

Article published online:
01 June 2022

© 2022. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

  • References

  • 1 Middleton B, Sittig DF, Wright A. Clinical decision support: a 25 year retrospective and a 25 year vision. Yearb Med Inform 2016; 25 (Suppl. 01) S103-S116
  • 2 Nanji KC, Garabedian PM, Shaikh SD. et al. Development of a perioperative medication-related clinical decision support tool to prevent medication errors: an analysis of user feedback. Appl Clin Inform 2021; 12 (05) 984-995
  • 3 Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med 2020; 3 (01) 17
  • 4 Osheroff JA, Teich J, Levick D. et al. Improving Outcomes with Clinical Decision Support: An Implementer's Guide. 2nd ed. Boca Raton, FL: CRC Press; 2012
  • 5 Shortliffe EH, Sepúlveda MJ. Clinical decision support in the era of artificial intelligence. JAMA 2018; 320 (21) 2199-2200
  • 6 Van Dort BA, Zheng WY, Sundar V, Baysari MT. Optimizing clinical decision support alerts in electronic medical records: a systematic review of reported strategies adopted by hospitals. J Am Med Inform Assoc 2021; 28 (01) 177-183
  • 7 Unberath P, Prokosch HU, Kraus S. A REST service for the visualization of clinical time series data in the context of clinical decision support. Stud Health Technol Inform 2019; 258: 26-30
  • 8 Berner ES, La Lande TJ. Overview of clinical decision support systems. In: Berner ES. ed. Clinical Decision Support Systems. New York, NY: Springer; 2007: 3-22
  • 9 Beasley JW, Wetterneck TB, Temte J. et al. Information chaos in primary care: implications for physician performance and patient safety. J Am Board Fam Med 2011; 24 (06) 745-751
  • 10 Schaefer E, Werner N, Scanlon M. Information chaos in the electronic health record as a threat to patient safety. In: Stephanidis C, Antona M, Ntoa S. eds. HCI International 2021 - Late Breaking Posters. Vol 1499. Communications in Computer and Information Science. New York, NY: Springer International Publishing; 2021: 189-196
  • 11 Kroth PJ, Morioka-Douglas N, Veres S. et al. Association of electronic health record design and use factors with clinician stress and burnout. JAMA Netw Open 2019; 2 (08) e199609
  • 12 Harle CA, Marlow NM, Schmidt SOF. et al. The effect of EHR-integrated patient-reported outcomes on satisfaction with chronic pain care. Am J Manag Care 2016; 22 (12) e403-e408
  • 13 O'Connor PJ, Sperl-Hillen JM, Rush WA. et al. Impact of electronic health record clinical decision support on diabetes care: a randomized trial. Ann Fam Med 2011; 9 (01) 12-21
  • 14 Bhat S, Derington CG, Trinkley KE. Clinicians' values and preferences for medication adherence and cost clinical decision support in primary care: a qualitative study. Appl Clin Inform 2020; 11 (03) 405-414
  • 15 Dorr DA, D'Autremont C, Pizzimenti C. et al. Assessing data adequacy for high blood pressure clinical decision support: a quantitative analysis. Appl Clin Inform 2021; 12 (04) 710-720
  • 16 Brunner MC, Sheehan SE, Yanke EM. et al. Joint design with providers of clinical decision support for value-based advanced shoulder imaging. Appl Clin Inform 2020; 11 (01) 142-152
  • 17 Fried LP. America's health and health care depend on preventing chronic disease. Huffington Post. Published March 8, 2017. Accessed March 20, 2020 at:
  • 18 Chronic Care. A call to action for health reform. Accessed March 20, 2020 at:
  • 19 Østbye T, Yarnall KS, Krause KM, Pollak KI, Gradison M, Michener JL. Is there time for management of patients with chronic diseases in primary care?. Ann Fam Med 2005; 3 (03) 209-214
  • 20 Tai-Seale M, Bolin J, Bao X, Street R. Management of chronic pain among older patients: inside primary care in the US. Eur J Pain 2011; 15 (10) 1087.e1-1087.e8
  • 21 Tsiga E, Panagopoulou E, Sevdalis N, Montgomery A, Benos A. The influence of time pressure on adherence to guidelines in primary care: an experimental study. BMJ Open 2013; 3 (04) e002700
  • 22 Freedman S, Golberstein E, Huang TY, Satin DJ, Smith LB. Docs with their eyes on the clock? The effect of time pressures on primary care productivity. J Health Econ 2021; 77: 102442
  • 23 Oakes AH, Adusumalli S, Snider CK, Rareshide CAL, Patel MS. Variation in cardiologist statin prescribing by clinic appointment time. J Am Coll Cardiol 2021; 77 (05) 661-662
  • 24 Neprash HT, Barnett ML. Association of primary care clinic appointment time with opioid prescribing. JAMA Netw Open 2019; 2 (08) e1910373
  • 25 McDonagh MS, Selph SS, Buckley DI. et al. Nonopioid Pharmacologic Treatments for Chronic Pain. Rockville, MD: Agency for Healthcare Research and Quality (US);. 2020 . Accessed February 23, 2022 at:
  • 26 Miaskowski C, Blyth F, Nicosia F. et al. A biopsychosocial model of chronic pain for older adults. Pain Med 2020; 21 (09) 1793-1805
  • 27 Gatchel RJ, Peng YB, Peters ML, Fuchs PN, Turk DC. The biopsychosocial approach to chronic pain: scientific advances and future directions. Psychol Bull 2007; 133 (04) 581-624
  • 28 Bendtsen P, Hensing G, Ebeling C, Schedin A. What are the qualities of dilemmas experienced when prescribing opioids in general practice?. Pain 1999; 82 (01) 89-96
  • 29 Bergman AA, Matthias MS, Coffing JM, Krebs EE. Contrasting tensions between patients and PCPs in chronic pain management: a qualitative study. Pain Med 2013; 14 (11) 1689-1697
  • 30 Dobscha SK, Corson K, Perrin NA. et al. Collaborative care for chronic pain in primary care: a cluster randomized trial. JAMA 2009; 301 (12) 1242-1252
  • 31 Ho KY, Gwee KA, Cheng YK, Yoon KH, Hee HT, Omar AR. Nonsteroidal anti-inflammatory drugs in chronic pain: implications of new data for clinical practice. J Pain Res 2018; 11: 1937-1948
  • 32 Harle CA, Apathy NC, Cook RL. et al. Information needs and requirements for decision support in primary care: an analysis of chronic pain care. AMIA Annu Symp Proc 2018; 2018: 527-534
  • 33 Marcial LH, Richardson JE, Lasater B. et al. The imperative for patient-centered clinical decision support. EGEMS (Wash DC) 2018; 6 (01) 12
  • 34 Marcial LH, Blumenfeld B, Harle C. et al. Barriers, facilitators, and potential solutions to advancing interoperable clinical decision support: multi-stakeholder consensus recommendations for the opioid use case. AMIA Annu Symp Proc 2020; 2019: 637-646
  • 35 Diiulio J, Militello LG, Andraka-Christou BT. et al. Factors that influence changes to existing chronic pain management plans. J Am Board Fam Med 2020; 33 (01) 42-50
  • 36 O'Brien EM, Staud RM, Hassinger AD. et al. Patient-centered perspective on treatment outcomes in chronic pain. Pain Med 2010; 11 (01) 6-15
  • 37 Becker KL. Tell me your dreams and goals: structuring communication exchanges to improve patient-centered care with chronic pain patients. Appl Nurs Res 2020; 53: 151248
  • 38 Harle CA, DiIulio J, Downs SM. et al. Decision-centered design of patient information visualizations to support chronic pain care. Appl Clin Inform 2019; 10 (04) 719-728
  • 39 Militello LG, Hurley RW, Cook RL. et al. Primary care clinicians' beliefs and strategies for managing chronic pain in an era of a national opioid epidemic. J Gen Intern Med 2020; 35 (12) 3542-3548
  • 40 Trafton JA, Martins SB, Michel MC. et al. Designing an automated clinical decision support system to match clinical practice guidelines for opioid therapy for chronic pain. Implement Sci 2010; 5 (01) 26
  • 41 Meadows G, Moesel C, Sebastian S, Winters D. The MITRE Corporation. Factors to Consider in Managing Chronic Pain: A Pain Management Summary. 2018 . Accessed February 23, 2021 at:
  • 42 Harle CA, Listhaus A, Covarrubias CM. et al. Overcoming barriers to implementing patient-reported outcomes in an electronic health record: a case report. J Am Med Inform Assoc 2016; 23 (01) 74-79
  • 43 Belden JL, Grayson R, Barnes J. Defining and testing EMR usability: principles and proposed methods of EMR usability evaluation and rating. . Published 2009 at:
  • 44 Ericsson KA, Simon HA. Verbal reports as data. Psychol Rev 1980; 87 (03) 215-251
  • 45 Kushniruk AW. Analysis of complex decision-making processes in health care: cognitive approaches to health informatics. J Biomed Inform 2001; 34 (05) 365-376
  • 46 Kushniruk A. Evaluation in the design of health information systems: application of approaches emerging from usability engineering. Comput Biol Med 2002; 32 (03) 141-149
  • 47 Nielsen J. Why you only need to test with 5 users. Published online March 18, 2000. Accessed February 23, 2022 at:
  • 48 Hwang W, Salvendy G. Number of people required for usability evaluation: the 10 ±  2 rule. Commun ACM 2010; 53 (05) 130-133
  • 49 Matthias MS, Talib TL, Huffman MA. Managing chronic pain in an opioid crisis: what is the role of shared decision-making?. Health Commun 2020; 35 (10) 1239-1247
  • 50 Vest JR, Ancker JS. Health information exchange in the wild: the association between organizational capability and perceived utility of clinical event notifications in ambulatory and community care. J Am Med Inform Assoc 2017; 24 (01) 39-46
  • 51 D'Amore J, Bouhaddou O, Mitchell S. et al. Interoperability progress and remaining data quality barriers of certified health information technologies. AMIA Annu Symp Proc 2018; 2018: 358-367
  • 52 van der Bij S, Khan N, Ten Veen P, de Bakker DH, Verheij RA. Improving the quality of EHR recording in primary care: a data quality feedback tool. J Am Med Inform Assoc 2017; 24 (01) 81-87
  • 53 Sayeed R, Gottlieb D, Mandl KD. SMART Markers: collecting patient-generated health data as a standardized property of health information technology. NPJ Digit Med 2020; 3 (01) 9
  • 54 Kharrazi H, Anzaldi LJ, Hernandez L. et al. The value of unstructured electronic health record data in geriatric syndrome case identification. J Am Geriatr Soc 2018; 66 (08) 1499-1507
  • 55 Naseri H, Kafi K, Skamene S. et al. Development of a generalizable natural language processing pipeline to extract physician-reported pain from clinical reports: Generated using publicly-available datasets and tested on institutional clinical reports for cancer patients with bone metastases. J Biomed Inform 2021; 120: 103864
  • 56 Trinkley KE, Blakeslee WW, Matlock DD. et al. Clinician preferences for computerised clinical decision support for medications in primary care: a focus group study. BMJ Health Care Inform 2019;26(01):0
  • 57 Peute LW, Aarts J, Bakker PJM, Jaspers MWM. Anatomy of a failure: a sociotechnical evaluation of a laboratory physician order entry system implementation. Int J Med Inform 2010; 79 (04) e58-e70
  • 58 Maviglia SM, Zielstorff RD, Paterno M, Teich JM, Bates DW, Kuperman GJ. Automating complex guidelines for chronic disease: lessons learned. J Am Med Inform Assoc 2003; 10 (02) 154-165
  • 59 Greenes RA, Bates DW, Kawamoto K, Middleton B, Osheroff J, Shahar Y. Clinical decision support models and frameworks: Seeking to address research issues underlying implementation successes and failures. J Biomed Inform 2018; 78: 134-143
  • 60 Berg M, Aarts J, van der Lei J. ICT in health care: sociotechnical approaches. Methods Inf Med 2003; 42 (04) 297-301
  • 61 Miller K, Mosby D, Capan M. et al. Interface, information, interaction: a narrative review of design and functional requirements for clinical decision support. J Am Med Inform Assoc 2018; 25 (05) 585-592
  • 62 Kannry J, McCullagh L, Kushniruk A, Mann D, Edonyabo D, McGinn T. A framework for usable and effective clinical decision support: experience from the iCPR randomized clinical trial. EGEMS (Wash DC) 2015; 3 (02) 1150
  • 63 Shneiderman B. The eyes have it: a task by data type taxonomy for information visualizations. In: Bederson BB, Shneiderman B. eds. The Craft of Information Visualization, Interactive Technologies. San Francisco, CA: Morgan Kaufmann; 2003: 364-371