Download PDF
Methods Inf Med 2008; 47(03): 251-259
DOI: 10.3414/ME0512
Original Article
Schattauer GmbH

Prioritizing the Risk Factors Influencing the Success of Clinical Information System Projects

A Delphi Study in Canada
G. Paré
1   Canada Research Chair in Information Technology in Health Care, HEC Montréal, Montreal, Quebec, Canada
,
C. Sicotte
2   University of Montréal, Montreal, Quebec, Canada
,
M. Jaana
3   University of Ottawa, Ottawa, Ontario, Canada
,
D. Girouard
1   Canada Research Chair in Information Technology in Health Care, HEC Montréal, Montreal, Quebec, Canada
› Author Affiliations
Further Information

Publication History

Received: 25 October 2007

accepted: 30 January 2007

Publication Date:
18 January 2018 (online)

    Permissions and Reprints

Summary

Objective: The aim of this study is to gain a better understanding of the risk factors influencing the success of clinical information system projects.

Methods: This study addresses this issue by first reviewing the extant literature on information technology project risks, and second conducting a Delphi survey among 21 experts highly involved in clinical information system projects in Québec, Canada, a region where government have invested heavily in health information technologies in recent years.

Results: Twenty-three risk factors were identified. The absence of a project champion was the factor that experts felt most deserves their attention. Lack of commitment from upper management was ranked second. Our panel of experts also confirmed the importance of a variable that has been extensively studied in information systems, namely, perceived usefulness that ranked third. Respondents ranked project ambiguity fourth. The fifth-ranked risk was associated with poor alignment between the clinical information systems’ characteristics and the organization of clinical work. The large majority of risk factors associated with the technology itself were considered less important. This finding supports the idea that technology-associated factors rarely figure among the main reasons for a project failure.

Conclusions: In addition to providing a comprehensive list of risk factors and their relative importance, the study presents a major contribution by unifying the literature on information systems and medical infor - matics. Our checklist provides a basis for further research that may help practitioners identify the effective countermeasures for mitigating risks associated with the implementation of clinical information systems.

Keywords

Clinical information systems - information technology - risk factors - project success - Delphi
 

  • References

  • 1 Ferratt TW, Lederer AL, Hall SR, Krella JM. Surmounting Health Information Network Barriers. The Greater Dayton Area Experience, Health Care Manage Rev 1998; 23 (01) 70-76.
    PubMedGoogle Scholar
  • 2 Payton FC, Ginzberg MJ. Inter-organizational Health Care Systems Implementations. An Exploratory Study of Early Electronic Commerce Initiatives, Health Care Manage Rev 2001; 26 (02) 20-32.
    PubMedGoogle Scholar
  • 3 Paré G, Jaana M, Sicotte C. Systematic Review of Home Telemonitoring for Chronic Diseases. The Evidence Base, J Am Med Inform Assoc 2007; 14 (03) 269-277.
    PubMedGoogle Scholar
  • 4 Chaudhry B, Wang J, Wu S, Maglione M, Mojica W, Roth E, Morton SC, Shekelle PG. Systematic Review: Impact of Health Information Technology on quality, efficiency, and costs of Medical Care. Ann Intern Med 2006; 144: E12-E22.
    PubMedGoogle Scholar
  • 5 Overhage JM, Perkins S, Tierney WM, McDonald CJ. Controlled Trial of Direct Physician Order Entry: Effects on Physicians’ Time Utilization in Ambulatory Care Internal Medicine Practices. J Am Med Inform Assoc 2001; 8 (04) 361-371.
    CrossrefPubMedGoogle Scholar
  • 6 Lepanto L, Paré G, Aubry D, Robillard P, Lesage J. Impact of PACS on Dictation Turnaround Time and Productivity. J Digit Imaging 2006; 19 (01) 92-97.
    CrossrefPubMedGoogle Scholar
  • 7 Kuhn KA, Wurst SH, Bott OJ, Giuse DA. Expanding the Scope of Health Information Systems. Challenges and Developments. Methods Inf Med 2006; 45 (01) 43-52.
    PubMedGoogle Scholar
  • 8 Evans RS, Pestotnik SL, Classen DC, Clemmer TP, Weaver LK, Orme JF, Lloyd JF, Burke JP. A Computer-Assisted Management Program for Antibiotics and Other Antiinfective Agents. N Engl J Med 1998; 338 (04) 232-238.
    CrossrefPubMedGoogle Scholar
  • 9 Michalowski W, Slowinski R, Wilk S, Farion KJ, Pike J, Rubin S. Design and Development of a Mobile System for Supporting Emergency Triage. Methods Inf Med 2005; 44 (01) 14-24.
    Article in Thieme ConnectPubMedGoogle Scholar
  • 10 Ash JS, Sittig DF, Poon EG, Guappone K, Campbell E, Dykstra RH. The Extent and Importance of Unintended Consequences Related to Computerized Provider Entry. J Am Med Inform Assoc 2007; 14 (04) 415-423.
    CrossrefPubMedGoogle Scholar
  • 11 O’ Connell RT, Cho C, Shah N, Brown K, Shiffman RN. Take Note(s): Differential EHR Satisfaction with Two Implementations under One Roof. J Am Med Inform Assoc 2004; 11 (01) 43-49.
    CrossrefPubMedGoogle Scholar
  • 12 Kleinke JD. Dot-Gov: Market Failure and the Creation of a National Health Information Technology System. Health Aff (Millwood) 2005; 24 (05) 1246-1262.
    CrossrefPubMedGoogle Scholar
  • 13 Sicotte C, Paré G, Moreault MP, Paccioni A. A Risk Assessment of Two Interorganizational Clinical Information Systems. J Am Med Inform Assoc 2006; 13 (05) 557-566.
    CrossrefPubMedGoogle Scholar
  • 14 Kaplan B. Culture Counts: How Institutional Values Affect Computer Use. M D Comput 2000; 17 (01) 23-26.
    PubMedGoogle Scholar
  • 15 Ozdas A, Miller RA. Care Provider Order Entry (CPOE): A Perspective on Factors Leading to Success or to Failure. Methods Inf Med 2007; 46 (01) 128-137.
    PubMedGoogle Scholar
  • 16 Aarts J, Berg M. Same Systems, Different Outcomes – Comparing the Implementation of Computerized Physician Order Entry in two Dutch Hospitals. Methods Inf Med 2006; 45 (01) 53-61.
    Article in Thieme ConnectPubMedGoogle Scholar
  • 17 Lapointe L, Rivard S. Getting Physicians to Accept New Information Technology: Insights from Case Studies. CMAJ 2006; 174 (11) 1573-1578.
    CrossrefPubMedGoogle Scholar
  • 18 Sicotte C, Denis JL, Lehoux P. The Computer Based Patient Record: A Strategic Issue in Process Innovation. J Med Syst 1998; 22 (06) 431-443.
    CrossrefPubMedGoogle Scholar
  • 19 Paré G, Trudel MC. Knowledge Barriers to PACS Adoption and Implementation in Hospitals. Int J Med Inform 2007; 76 (01) 22-33.
    CrossrefPubMedGoogle Scholar
  • 20 Schmidt R, Lyytinen K, Keil M, Cule P. Identifying Software Project Risks: An International Delphi Study. J Manage Inform Syst 2001; 17 (04) 5-36.
    CrossrefPubMedGoogle Scholar
  • 21 Chiang MF, Starren JB. Software Engineering Risk Factors in the Implementation of a Small Electronic Medical Record System: The Problem of Scalability. Proc AMIA Symp 2002 pp 145-149.
    PubMedGoogle Scholar
  • 22 Brender J, Ammenwerth E, Nykanen P, Talmon J. Factors Influencing Success and Failure of Health Informatics Systems. Methods Inf Med 2006; 45 (01) 125-136.
    Article in Thieme ConnectPubMedGoogle Scholar
  • 23 Barki H, Rivard S, Talbot J. Toward an Assessment of Software Development Risk. J Manage Inform Syst 1993; 10 (02) 203-225.
    CrossrefPubMedGoogle Scholar
  • 24 Sherer SA, Alter S. Information System Risks and Risk Factors: Are they Mostly about Information Systems?. Commun AIS 2004; 14: 29-64.
    PubMedGoogle Scholar
  • 25 Doolan DF, Bates DW, James BC. The Use of Computers for Clinical Care: A Case Series of Advanced U S. Sites. J Am Med Inform Assoc 2003; 10 (01) 94-107.
    CrossrefPubMedGoogle Scholar
  • 26 Starr P. Smart Technology, Stunted Policy: Developing Health Information Networks. Health Aff (Millwood) 1997; 16 (03) 91-105.
    PubMedGoogle Scholar
  • 27 Overhage JM, Evans L, Marchibroda J. Communities’ Readiness for Health Information Exchange: The National Landscape in 2004. J Am Med Inform Assoc 2005; 12 (02) 107-112.
    CrossrefPubMedGoogle Scholar
  • 28 Markus ML. Power, Politics, and MIS Implementation. Commun ACM 1983; 26 (06) 430-444.
    CrossrefPubMedGoogle Scholar
  • 29 Lyytinen K, Hirschheim R. Information Systems Failures: A Survey and Classification of the Empirical Literature. Oxford Surv Inform Tech 1987; 4: 257-309.
    PubMedGoogle Scholar
  • 30 Battacherjee A. Understanding Information Systems Continuance: An Expectation-Confirmation Model. MIS Quart 2001; 25 (03) 351-370.
    CrossrefPubMedGoogle Scholar
  • 31 Miller RH, Sim I. Physicians’ Use of Electronic Medical Records: Barriers and Solutions. Health Aff (Millwood) 2004; 23 (02) 116-126.
    CrossrefPubMedGoogle Scholar
  • 32 Spil TAM, Schuring RW, Michel-Verkerke MB. Electronic Prescription System: Do Professionals Use It?. Int J Healthc Technol Manage 2004; 6 (01) 32-55.
    PubMedGoogle Scholar
  • 33 Lee F, Teich JM, Spurr CD, Bates DW. Implementation of Physician Order Entry: User Satisfaction and Self-reported Usage Patterns. J Am Med Inform Assoc 1996; 3: 42-55.
    CrossrefPubMedGoogle Scholar
  • 34 Ahmad A, Teater P, Bentley TD, Kuehn L, Kumar RR, Thomas A, Mekhjian HS. Key Attributes of a Successful Physician Order Entry System Implementation in a Multi-hospital Environment. J Am Med Inform Assoc 2002; 9: 16-24.
    CrossrefPubMedGoogle Scholar
  • 35 Van Der Meijden MJ, Tange HJ, Troost J, Hasman A. Determinants of Success of Inpatient Clinical Information Systems: A Literature Review. J Am Med Inform Assoc 2003; 10 (03) 235-243.
    CrossrefPubMedGoogle Scholar
  • 36 Siau K. Inter-organizational Systems and Competitive Advantages – Lessons from History. J Comput Inform Syst 2003; 44 (01) 33-39.
    PubMedGoogle Scholar
  • 37 Anderson JG, Aydin CE. Evaluating the Impact of Health Care Information Systems. Int J Technol Assess Health Care 1997; 13 (02) 380-393.
    CrossrefPubMedGoogle Scholar
  • 38 Hunsucker JL, Loos D. Transition Management: An Analysis of Strategic Considerations for Effective Implementation. Eng Manage Int 1989; 5: 167-178.
    PubMedGoogle Scholar
  • 39 Tiwana A, Keil M. The One-Minute Risk Assessment Tool. Commun ACM 2004; 47 (11) 73-77.
    PubMedGoogle Scholar
  • 40 Jarvenpaa SL, Ives B. Executive Involvement and Participation in the Management of Information Technology. MIS Quart 1991; 15 (02) 205-227.
    CrossrefPubMedGoogle Scholar
  • 41 Hazeltine FW. The Key to Successful Implementations. Prod Inv Manage R APICS NEWS 1990; 10 (11) 41-42 45.
    PubMedGoogle Scholar
  • 42 Schmidt R. Managing Delphi Surveys Using Nonparametric Statistical Techniques. Decision Sci 1997; 28 (03) 763-774.
    CrossrefPubMedGoogle Scholar
  • 43 Beath CM. Supporting the Information Technology Champion. MIS Quart 1991; 15 (03) 355-372.
    CrossrefPubMedGoogle Scholar
  • 44 Dong L, Sun H, Fang Y. Do Perceived Leadership Behaviors Affect User Technology Beliefs? An Examination of the Impact of Project Champions and Direct Managers. Commun AIS 2007; 19: 655-664.
    PubMedGoogle Scholar
  • 45 Mintzberg H. Structure et Dynamique des Organisations. Paris: Editions d’Organisation; 1982
    Google Scholar
  • 46 Paré G, Elam JJ, Ward CG. Implementation of a Patient Charting System: Challenges Encountered and Tactics Adopted in a Burn Center. J Med Syst 1997; 21 (01) 49-66.
    CrossrefPubMedGoogle Scholar
  • 47 Jeyeraj A, Rottman JW, Lacity MC. A Review of the Predictors, Linkages, and Biases in IT Innovation Adoption Research. J Inform Tech 2006; 21 (01) 1-23.
    PubMedGoogle Scholar
  • 48 Davis FD. Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Quart 1989; 13 (03) 318-339.
    PubMedGoogle Scholar
  • 49 Lorenzi NM, Riley RT. Organizational Aspects of Health Informatics. Managing Technological Change, New York, NY: Springer Verlag; 1995
    Google Scholar
  • 50 Kanter RM. The Enduring Skills of Change Leaders. Ivey Bus J 2000: 31-36.
    PubMedGoogle Scholar
  • 51 Kotter JP. Leading Change. Boston: Harvard University Press; 1996
    Google Scholar
  • 52 Shu K, Boyle D, Spurr C, Horsky J, Heiman H, O’ Connor P, Lepore J, Bates DW. Comparison of Time Spent Writing Orders on Paper with Computerized Physician Order Entry. Medinfo 2001; 10 Pt 2 1207-1211.
    PubMedGoogle Scholar
  • 53 Weiner M, Gress T, Thiemann DR, Jenckes M, Reel SI, Mandell SF, Bass EB. Contrasting Views of Physicians and Nurses about an Inpatient Computer- based Provider Order-entry System. J Am Med Inform Assoc 1999; 6 (03) 234-244.
    CrossrefPubMedGoogle Scholar
  • 54 Chiasson M, Reddy M, Kaplan B, Davidson E. Expanding Multi-disciplinary Approaches to Healthcare Information Technologies: What does Information Systems Offer Medical Informatics?. Int J Med Inform 2007; 76S: S89-S97.
    PubMedGoogle Scholar