Yearb Med Inform 2006; 15(01): 145-158
DOI: 10.1055/s-0038-1638486
Reviews
Georg Thieme Verlag KG Stuttgart

Computer-Interpretable Clinical Practice Guidelines

Where Are We and where Are We Going?
F. A. Sonnenberg
1   Department of Medicine, UMDNJ Robert Wood Johnson Medical School, New Brunswik, New Jersey, USA
,
C. G. Hagerty
1   Department of Medicine, UMDNJ Robert Wood Johnson Medical School, New Brunswik, New Jersey, USA
› Author Affiliations
Further Information

Publication History

Publication Date:
07 March 2018 (online)

Summary

Objectives

To provide a comprehensive overview of computerinterpretable guideline (CIG) systems aimed at non-experts. The overview includes the history of efforts to develop CIGs, features of and relationships among current major CIG systems, current status of standards developments pertinent to CIGs and identification of unsolved problems and needs for future research

Methods

Literature re view based on PubMed, AMIA conference proceedings and key references from publications identified. Search terms included practice guidelines, decision support, controlled vocabulary and medical record systems. Papers were reviewed by both authors and summarized narratively.

Results

There is a consensus that guideline delivery systems must be integrated with electronic health records (EHRs) to be most effective. Several evolving CIG formalisms have in common, use of a task network model. There is currently no dominant CIG system. The major challenge in development of interoperable CIGs, is agreement on a standard controlled vocabulary. Such standards are under development, but not widely used, particularly in commercial EHR systems. The Virtual Medical Record (VMR) concept has been proposed as a standard that would serve as an intermediary between guideline vocabulary and that used in EHR implementation.

Conclusion

CIG systems are in a state of evolution. Standards efforts promise to improve interoperability without compromising innovation. The VMR concept can assist guideline development even before clinical systems routinely adhere to standards. Frontiers for future work include using the principles learned by computer implementation of guidelines to improve the guideline development process and evaluation methods that isolate the effects of specific CIG features.

 
  • References

  • 1 Field MJ, Lohr KN. Institute of MCoCP, Guidelines. Guidelines for clinical practice from development to use. Washington, D.C: National Academy Press; 1992
  • 2 Eddy DM. Clinical decision making: from theory to practice. Practice policies—guidelines for methods. JAMA Apr 4 1990; 263 (13) 1839-41.
  • 3 Leape LL, Park RE, Solomon DH, Chassin MR, Kosecoff J, Brook RH. Does inappropriate use explain small-area variations in the use of health care services?. JAMA Feb 2 1990; 263 (05) 669-72.
  • 4 Ellrodt G, Cook DJ, Lee J, Cho M, Hunt D, Weingarten S. Evidence-based disease management. JAMA Nov 26 1997; 278 (20) 1687-92.
  • 5 Woolf SH. Practice guidelines: a new reality in medicine. I. Recent developments. Arch Intern Med 1990; 150 (09) 1811.
  • 6 Epstein A. Performance reports on quality— prototypes, problems, and prospects. N Engl J Med 1995; 333 (01) 57.
  • 7 Joint Commission on Accreditation of Healthcare Organizations: Specification Manual for National Hospital Quality Measures. 2005: 2005.
  • 8 Bodenheimer T. The American health care system— the movement for improved quality in health care. N Engl J Med Feb 11 1999; 340 (06) 488-92.
  • 9 Julian DG. Translation of clinical trials into clinical practice. J Intern Med Mar 2004; 255 (03) 309-16.
  • 10 McGlynn EA, Asch SM, Adams J. et al. The quality of health care delivered to adults in the United States. N Engl J Med 2003; 348 (26) 2635-45.
  • 11 Schuster MA, McGlynn EA, Brook RH. How good is the quality of health care in the United States?. Milbank Q 1998; 76 (04) 517-63 509.
  • 12 Grol R. Successes and failures in the implementation of evidence-based guidelines for clinical practice. Med Care Aug 2001; 39 (8 Suppl 2): II46-54.
  • 13 Schuster MA, McGlynn EA, Brook RH. How good is the quality of health care in the United States? 1998. Milbank Q 2005; 83 (04) 843-95.
  • 14 Brook RH. Practice guidelines and practicing medicine. Are they compatible? JAMA 1989; 262 (21) 3027.
  • 15 Graham ID, Calder LA, Hebert PC, Carter AO, Tetroe JM. A comparison of clinical practice guideline appraisal instruments. Int J Technol Assess Health Care Autumn 2000; 16 (04) 1024-38.
  • 16 Cluzeau F. Appraisal of Guidelines for Research & Evaluation; The AGREE Instrument. Vol 2006: 2001.
  • 17 Shiffman RN, Shekelle P, Overhage JM, Slutsky J, Grimshaw J, Deshpande AM. Standardized reporting of clinical practice guidelines: a proposal from the Conference on Guideline Standardization. Ann Intern Med Sep 16 2003; 139 (06) 493-8.
  • 18 Grimshaw JM, Russell IT. Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations. Lancet Nov 27 1993; 342 (8883): 1317-22.
  • 19 Worrall G, Chaulk P, Freake D. The effects of clinical practice guidelines on patient outcomes in primary care: a systematic review. CMAJ 1997; 156 (12) 1705-12.
  • 20 Shiffman RN, Liaw Y, Brandt CA, Corb GJ. Computer-based guideline implementation systems: a systematic review of functionality and effectiveness. J Am Med Inform Assoc 1999; 06 (02) 104-14.
  • 21 Woolf SH, Grol R, Hutchinson A, Eccles M, Grimshaw J. Clinical guidelines: potential benefits, limitations, and harms of clinical guidelines. BMJ Feb 20 1999; 318 (7182): 527-30.
  • 22 Andrews EJ, Redmond HP. A review of clinical guidelines. Br J Surg Aug 2004; 91 (08) 956-64.
  • 23 Lomas J, Anderson GM, Domnick-Pierre K, Vayda E, Enkin MW, Hannah WJ. Do practice guidelines guide practice? The effect of a consensus statement on the practice of physicians. N Engl J Med Nov 9 1989; 321 (19) 1306-11.
  • 24 Tunis SR, Hayward RS, Wilson MC, Rubin HR, Bass EB, Johnston M. et al. Internists’ attitudes about clinical practice guidelines. Ann Intern Med Jun 1 1994; 120 (11) 956-63.
  • 25 Ellrodt AG, Conner L, Riedinger M, Weingarten S. Measuring and improving physician compliance with clinical practice guidelines. A controlled interventional trial. Ann Intern Med 1995; 122 (04) 277-82.
  • 26 Cabana MD, Rand CS, Powe NR. et al. Why don’t physicians follow clinical practice guidelines? A framework for improvement. JAMA 1999; 282 (15) 1458-65.
  • 27 Jenders RA, Sailors RM. Convergence on a standard for representing clinical guidelines: work in health level seven. Medinfo 2004; 11 (Pt 1): 130-4.
  • 28 Executive Summary of The Third Report of The National Cholesterol Education Program (NCEP) Expert Panel on Detection, Evaluation, And Treatment of High Blood Cholesterol In Adults (Adult Treatment Panel III). JAMA 2001; 285 (19) 2486-97.
  • 29 Grundy SM, Cleeman JI, Merz CN, Brewer Jr HB, Clark LT, Hunninghake DB. et al. Implications of recent clinical trials for the National Cholesterol Education Program Adult Treatment Panel III guidelines. Circulation 2004; 110 (02) 227-39.
  • 30 Tierney WM, Hui SL, McDonald CJ. Delayed feedback of physician performance versus immediate reminders to perform preventive care. Effects on physician compliance. Med Care Aug 1986; 24 (08) 659-66.
  • 31 Weingarten SR, Riedinger MS, Conner L. et al. Practice guidelines and reminders to reduce duration of hospital stay for patients with chest pain. An interventional trial. Ann Intern Med Feb 15 1994; 120 (04) 257-63.
  • 32 Safran C, Rind DM, Davis RB. et al. Guidelines for management of HIV infection with computerbased patient’s record. Lancet Aug 5 1995; 346 (8971): 341-6.
  • 33 Zielstorff RD. Online practice guidelines: issues, obstacles, and future prospects. J Am Med Inform Assoc May-Jun 1998; 05 (03) 227-36.
  • 34 Subramanian U, Fihn SD, Weinberger M, Plue L, Smith FE, Udris EM. et al. A controlled trial of including symptom data in computer-based care suggestions for managing patients with chronic heart failure. Am J Med Mar 15 2004; 116 (06) 375-84.
  • 35 Vissers MC, Biert J, van der Linden CJ, Hasman A. Effects of a supportive protocol processing system (ProtoVIEW) on clinical behaviour of residents in the accident and emergency department. Comput Methods Programs Biomed 1996; 49 (02) 177-84.
  • 36 Rossi RA, Every NR. A computerized intervention to decrease the use of calcium channel blockers in hypertension. J Gen Intern Med 1997; 12 (11) 672-8.
  • 37 Schriger DL, Baraff LJ, Rogers WH, Cretin S. Implementation of clinical guidelines using a computer charting system. Effect on the initial care of health care workers exposed to body fluids. JAMA Nov 19 1997; 278 (19) 1585-90.
  • 38 Rollman BL, Gilbert T, Lowe HJ, Kapoor WN, Schulberg HC. The electronic medical record: its role in disseminating depression guidelines in primary care practice. Int J Psychiatry Med 1999; 29 (03) 267-86.
  • 39 Cannon DS, Allen SN. A comparison of the effects of computer and manual reminders on compliance with a mental health clinical practice guideline. J Am Med Inform Assoc 2000; 07 (02) 196-203.
  • 40 Schriger DL, Baraff LJ, Buller K. et al. Implementation of clinical guidelines via a computer charting system: effect on the care of febrile children less than three years of age. J Am Med Inform Assoc Mar-Apr 2000; 07 (02) 186-95.
  • 41 Rollman BL, Hanusa BH, Gilbert T, Lowe HJ, Kapoor WN, Schulberg HC. The electronic medical record. A randomized trial of its impact on primary care physicians’ initial management of major depression [corrected]. Arch Intern Med Jan 22 2001; 161 (02) 189-97.
  • 42 Ornstein SM. Translating research into practice using electronic medical records the PPRNet-TRIP project: primary and secondary prevention of coronary heart disease and stroke. Top Health Inf Manage Nov 2001; 22 (02) 52-8.
  • 43 Maviglia SM, Teich JM, Fiskio J, Bates DW. Using an electronic medical record to identify opportunities to improve compliance with cholesterol guidelines. J Gen Intern Med Aug 2001; 16 (08) 531-7.
  • 44 Tierney WM, Overhage JM, Murray MD, Harris LE, Zhou XH, Eckert GJ. et al. Effects of computerized guidelines for managing heart disease in primary care. J Gen Intern Med 2003; 18 (12) 967-76.
  • 45 Montgomery AA, Fahey T, Peters TJ, MacIntosh C, Sharp DJ. Evaluation of computer based clinical decision support system and risk chart for management of hypertension in primary care: randomised controlled trial. BMJ Mar 11 2000; 320 (7236): 686-90.
  • 46 Ansari M, Shlipak MG, Heidenreich PA. et al. Improving guideline adherence: a randomized trial evaluating strategies to increase beta-blocker use in heart failure. Circulation Jun 10 2003; 107 (22) 2799-804.
  • 47 Montgomery AA, Peters TJ, Fahey T. Reasons physicians do not follow clinical practice guidelines. JAMA 2000; 283 (13) 1685 author reply 1686..
  • 48 Peleg M, Tu S, Bury J, Ciccarese P, Fox J, Greenes RA. et al. Comparing computer-interpretable guideline models: a case-study approach. J Am Med Inform Assoc Jan-Feb 2003; 10 (01) 52-68.
  • 49 Wetter T. Lessons learnt from bringing knowledgebased decision support into routine use. Artif Intell Med Mar 2002; 24 (03) 195-203.
  • 50 Shiffman RN. Representation of clinical practice guidelines in conventional and augmented decision tables. J Am Med Inform Assoc Sep-Oct 1997; 04 (05) 382-93.
  • 51 Shaneyfelt TM, Mayo-Smith MF, Rothwangl J. Are guidelines following guidelines? The methodological quality of clinical practice guidelines in the peer-reviewed medical literature. JAMA May 26 1999; 281 (20) 1900-5.
  • 52 Fretheim A, Williams Jr. JW, Oxman AD, Herrin J. The relation between methods and recommendations in clinical practice guidelines for hypertension and hyperlipidemia. J Fam Pract 2002; 51 (11) 963-8.
  • 53 Sonnenberg F, Hagerty CG, Acharya J, Pickens D, Kulikowski C. Vocabulary Requirements for Implementing Clinical Guidelines in an Electronic Medical Record: A Case Study. Proc AMIA Annu Fall Symp 2005; 709-13.
  • 54 Major outcomes in high-risk hypertensive patients randomized to angiotensin-converting enzyme inhibitor or calcium channel blocker vs diuretic: The Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack Trial (ALLHAT). Jama Dec 18 2002; 288 (23) 2981-2997.
  • 55 Chobanian AV, Bakris GL, Black HR, Cushman WC, Green LA, Izzo Jr JL. et al. The Seventh Report of the Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure: the JNC 7 report. JAMA 2003; 289 (19) 2560-72.
  • 56 Codish S, Shiffman RN. A Model of Ambiguity and Vagueness in Clinical Practice Guideline Recommendations. Proc AMIA Annu Fall Symp 2005; 146-50.
  • 57 Shiffman RN, Michel G, Essaihi A, Thornquist E. Bridging the guideline implementation gap: a systematic, document-centered approach to guideline implementation. J Am Med Inform Assoc Sep-Oct 2004; 11 (05) 418-26.
  • 58 Wang D, Peleg M, Tu SW, Boxwala AA, Greenes RA, Patel VL. et al. Representation primitives, process models and patient data in computerinterpretable clinical practice guidelines: a literature review of guideline representation models. Int J Med Inform Dec 18 2002; 68 (1-3): 59-70.
  • 59 de Clercq PA, Blom JA, Korsten HH, Hasman A. Approaches for creating computer-interpretable guidelines that facilitate decision support. Artif Intell Med May 2004; 31 (01) 1-27.
  • 60 Hripcsak G, Cimino JJ, Johnson SB, Clayton PD. The Columbia-Presbyterian Medical Center decision-support system as a model for implementing the Arden Syntax. Proc Annu Symp Comput Appl Med Care 1991; 248-52.
  • 61 Hripcsak G, Wigertz OB, Kahn MG, Clayton PD, Pryor TA. ASTM E31.15 on health knowledge representation: the Arden Syntax. Stud Health Technol Inform 1993; 06: 105-12.
  • 62 Wang D. Translating Arden MLMs into GLIF guidelines—a case study of hyperkalemia patient screening. Stud Health Technol Inform 2004; 101: 177-81.
  • 63 Agrawal A, Shiffman RN. Using GEM-encoded guidelines to generate medical logic modules. Proc AMIA Symp 2001; 7-11.
  • 64 Pryor TA, Hripcsak G. Sharing MLM’s: an experiment between Columbia-Presbyterian and LDS Hospital. Proc Annu Symp Comput Appl Med Care 1993; 399.
  • 65 Musen MA, Tu SW, Das AK, Shahar Y. EON: a component-based approach to automation of protocol-directed therapy. J Am Med Inform Assoc Nov-Dec 1996; 03 (06) 367-88.
  • 66 Johnson PD, Tu SW, Musen MA, Purves I. A virtual medical record for guideline-based decision support. Proc AMIA Symp 2001; 294-8.
  • 67 Fox J, Johns N, Lyons C, Rahmanzadeh A, Thomson R, Wilson P. PROforma: a general technology for clinical decision support systems. Comput Methods Programs Biomed Sep 1997; 54 1-2: 59-67.
  • 68 Shahar Y, Miksch S, Johnson P. The Asgaard project: a task-specific framework for the application and critiquing of time-oriented clinical guidelines. Artif Intell Med Sep-Oct 1998; 14 (1-2): 29-51.
  • 69 Shahar Y, Young O, Shalom E, Galperin M, Mayaffit A, Moskovitch R. et al. A framework for a distributed, hybrid, multiple-ontology clinicalguideline library, and automated guideline-support tools. J Biomed Inform 2004; 37 (05) 325-44.
  • 70 Quaglini S, Stefanelli M, Lanzola G, Caporusso V, Panzarasa S. Flexible guideline-based patient careflow systems. Artif Intell Med Apr 2001; 22 (01) 65-80.
  • 71 Quaglini S, Stefanelli M, Cavallini A, Micieli G, Fassino C, Mossa C. Guideline-based careflow systems. Artif Intell Med Aug 2000; 20 (01) 5-22.
  • 72 Ohno-Machado L, Gennari JH, Murphy SN, Jain NL, Tu SW, Oliver DE. et al. The guideline interchange format: a model for representing guidelines. J Am Med Inform Assoc 1998; 05 (04) 357.
  • 73 Patel VL, Allen VG, Arocha JF, Shortliffe EH. Representing clinical guidelines in GLIF: individual and collaborative expertise. J Am Med Inform Assoc Sep-Oct 1998; 05 (05) 467-83.
  • 74 Boxwala AA, Peleg M, Tu S, Ogunyemi O, Zeng QT, Wang D, Patel VL. et al. GLIF3: a representation format for sharable computer-interpretable clinical practice guidelines. J Biomed Inform Jun 2004; 37 (03) 147-61.
  • 75 Brickley DaGRV. Resource Description Framework (RDF) Schema Specification 1.0, W3C Candidate Recommendation. Boston: World Wide WebConsortium; 2000
  • 76 Sordo M, Ogunyemi O, Boxwala AA, Greenes RA. GELLO: an object-oriented query and expression language for clinical decision support. AMIA Annu Symp Proc 2003; 1012.
  • 77 Wang D, Shortliffe EH. GLEE—a model-driven execution system for computer-based implementation of clinical practice guidelines. Proc AMIA Symp 2002; 855-9.
  • 78 Campbell J, Tu S, Mansfield JG. et al. The SAGE Guideline Model: A Knowledge Representation Framework for Encoding Interoperable Clinical Practice Guidelines SMI Technical Report 2003-0962. 2003
  • 79 Tu SW, Campbell JR, Musen MA. SAGE Guideline Modeling: Motivation and Methodology Paper presented at: Computer-Based Support for Clinical Guidelines and Protocols: Proceedings of the Symposium on Computerized Guidelines and Protocols. 2004
  • 80 Goldstein MK, Hoffman BB, Coleman RW, Musen MA, Tu SW, Advani A. et al. Implementing clinical practice guidelines while taking account of changing evidence: ATHENA DSS, an easily modifiable decision-support system for managing hypertension in primary care. Proc AMIA Symp 2000; 300-4.
  • 81 Goldstein MK, Coleman RW, Tu SW, Shankar RD, O’Connor MJ, Musen MA. et al. Translating research into practice: organizational issues in implementing automated decision support for hypertension in three medical centers. J Am Med Inform Assoc Sep-Oct 2004; 11 (05) 368-76.
  • 82 Tu SM MA, Shankara R, Campbell J, Hrabak K, McClayb J, Huffc SM. et al. Modeling Guidelines for Integration into Clinical Workflow. Proc Medinfo 2004; 174-8.
  • 83 Herbert SI, Gordon CJ, Jackson-Smale A, Salis JL. Protocols for clinical care. Comput Methods Programs Biomed Sep-Oct 1995; 48 (1-2): 21-6.
  • 84 De Clercq PA, Blom JA, Hasman A, Korsten HH. GASTON: an architecture for the acquisition and execution of clinical guideline-application tasks. Med Inform Internet Med Oct-Dec 2000; 25 (04) 247-63.
  • 85 Votruba P, Miksch S, Kosara R. Facilitating knowledge maintenance of clinical guidelines and protocols. Medinfo 2004; 11 (Pt 1): 57-61.
  • 86 Shiffman RN, Agrawal A, Deshpande AM, Gershkovich P. An approach to guideline implementation with GEM. Medinfo 2001; 10 (Pt 1): 271-5.
  • 87 Shiffman RN, Karras BT, Agrawal A, Chen R, Marenco L, Nath S. GEM: a proposal for a more comprehensive guideline document model using XML. J Am Med Inform Assoc 2000; 07 (05) 488-98.
  • 88 Shiffman RN, Michel G. Toward improved guideline quality: using the COGS statement with GEM. Medinfo 2004; 11 (Pt 1): 159-63.
  • 89 Tang PC, Young CY. ActiveGuidelines: integrating Web-based guidelines with computer-based patient records. Proc AMIA Symp 2000; 843-7.
  • 90 Hagerty CG, Pickens D, Kulikowski C, Sonnenberg F. HGML: a hypertext guideline markup language. Proc AMIA Symp 2000; 325-9.
  • 91 Eriksson HT, SWMusen M. Semantic Clinical Guideline Documents. Proc AMIA Annu Fall Symp 2005; 236-40.
  • 92 Shankar RD, Tu SW, Martins SB, Fagan LM, Goldstein MK, Musen MA. Integration of textual guideline documents with formal guideline knowledge bases. Proc AMIA Symp 2001; 617-21.
  • 93 Hagerty CC J, Pickens DS, Kulikowski CA, Sonnenberg FA. Semi-Automated Encoding of Guidelines. Proc Medinfo 2004; 1625.
  • 94 Georg G, Jaulent M. An Environment for Document Engineering of Clinical Guidelines. AMIA Annu Symp Proc 2005; 276-80.
  • 95 7” HL. HL 7 Reference Information Model. Web Page]. http://www.hl7.org/Library/data-model/RIM/ modelpage_mem.htm Accessed 1/12/06..
  • 96 Jenders RA, Sujansky W, Broverman CA, Chadwick M. Towards improved knowledge sharing: assessment of the HL7 Reference Information Model to support medical logic module queries. Proc AMIA Annu Fall Symp 1997; 308-12.
  • 97 Schadow G, Russler DC, Mead CN, McDonald CJ. Integrating medical information and knowledge in the HL7 RIM. Proc AMIA Symp 2000; 764-8.
  • 98 Johnson PD, Tu S, Booth N, Sugden B, Purves IN. Using scenarios in chronic disease management guidelines for primary care. Proc AMIA Symp 2000; 389-93.
  • 99 National Heart LaBI. Expert Panel Report 2: guidelines for the diagnosis and management of asthma. SOURCE(S): U.S. Department of Health and Human Services. 1997
  • 100 Mandell LA, Bartlett JG, Dowell SF, File Jr. TM, Musher DM, Whitney C. Update of practice guidelines for the management of communityacquired pneumonia in immunocompetent adults. Clin Infect Dis Dec 1 2003; 37 (11) 1405-33.
  • 101 Sanders GD, Nease Jr. RF, Owens DK. Publishing web-based guidelines using interactive decision models. J Eval Clin Pract May 2001; 07 (02) 175-89.
  • 102 Lindberg DA, Humphreys BL, McCray AT. The Unified Medical Language System. Methods Inf Med 1993; 32 (04) 281.
  • 103 Wang AY, Sable JH, Spackman KA. The SNOMED clinical terms development process: refinement and analysis of content. Proc AMIA Symp 2002; 845-9.
  • 104 McDonald CJ, Huff SM, Suico JG. et al. LOINC, a universal standard for identifying laboratory observations: a 5-year update. Clin Chem Apr 2003; 49 (04) 624-33.
  • 105 Peleg M, Boxwala AA, Bernstam E, Tu S, Greenes RA, Shortliffe EH. Sharable representation of clinical guidelines in GLIF: relationship to the Arden Syntax. J Biomed Inform Jun 2001; 34 (03) 170-81.
  • 106 Waitman LR, Miller RA. Pragmatics of implementing guidelines on the front lines. J Am Med Inform Assoc Sep-Oct 2004; 11 (05) 436-8.
  • 107 Stead WW, Kelly BJ, Kolodner RM. Achievable steps toward building a National Health Information infrastructure in the United States. J Am Med Inform Assoc Mar-Apr 2005; 12 (02) 113-20.
  • 108 Matchar DB. McCormick Ka MS RaSRA. Application of Decision Analysis to Guideline Development. Clinical Practice Guideline Development: Methodology Perspectives;Agency for Health Care Policy and Research Publication No. 95-00009: U.S. Department of Health and Human Services. 1995
  • 109 Boyd CM, Darer J, Boult C, Fried LP, Boult L, Wu AW. Clinical practice guidelines and quality of care for older patients with multiple comorbid diseases: implications for pay for performance. JAMA Aug 10 2005; 294 (06) 716-24.
  • 110 O’Connor PJ. Adding value to evidence-based clinical guidelines. JAMA Aug 10 2005; 294 (06) 741-3.
  • 111 Shiffman RN, Dixon J, Brandt C, Essaihi A, Hsiao A, Michel G. et al. The GuideLine Implementability Appraisal (GLIA): development of an instrument to identify obstacles to guideline implementation. BMC Med Inform Decis Mak Jul 27 2005; 05: 23.
  • 112 Sutton DR, Fox J. The syntax and semantics of the PROforma guideline modeling language. J Am Med Inform Assoc 2003; 10 (05) 433-43.
  • 113 Ciccarese P, Caffi E, Quaglini S, Stefanelli M. Architectures and tools for innovative Health Information Systems: the Guide Project. Int J Med Inform Aug 2005; 74 (7-8): 553-62.