State of the Art in Clinical Informatics: Evidence and Examples

Summary

Objective: The field of clinical informatics has expanded substantially in the six decades since its inception. Early research focused on simple demonstrations that health information technology (HIT) such as electronic health records (EHRs), computerized provider order entry (CPOE), and clinical decision support (CDS) systems were feasible and potentially beneficial in clinical practice.

Methods: In this review, we present recent evidence on clinical informatics in the United States covering three themes: 1) clinical informatics systems and interventions for providers, including EHRs, CPOE, CDS, and health information exchange; 2) consumer health informatics systems, including personal health records and web-based and mobile HIT; and 3) methods and governance for clinical informatics, including EHR usability; data mining, text mining, natural language processing, privacy, and security.

Results: Substantial progress has been made in demonstrating that various clinical informatics methodologies and applications improve the structure, process, and outcomes of various facets of the healthcare system.

Conclusion: Over the coming years, much more will be expected from the field. As we move past the “early adopters” in Rogers' diffusion of innovations' curve through the “early majority” and into the “late majority,” there will be a crucial need for new research methodologies and clinical applications that have been rigorously demonstrated to work (i.e., to improve health outcomes) in multiple settings with different types of patients and clinicians.

Keywords

Informatics - evidence-based practice - medical order entry systems - decision support systems - clinical - data mining

 

• References

• 1 Selby JV, Beal AC, Frank L. The Patient-Centered Outcomes Research Institute (PCORI) national priorities for research and initial research agenda. JAMA: J Am Med Assoc 2012; 307 (15) 1583-4.
• 2 Stead WW, Haynes RB, Fuller S, Friedman CP, Travis LE, Beck JR. et al. Designing medical informatics research and library-resource projects to increase what is learned. J Am Med Inform Assoc 1994; Feb 1 (1) 28-33.
• 3 Jha AK, DesRoches CM, Campbell EG, Donelan K, Rao SR, Ferris TG. et al. Use of electronic health records in U.S. hospitals. N Engl J Med 2009; Apr 16 360 (16) 1628-38.
• 4 Bates DW, Cullen DJ, Laird N, Petersen LA, Small SD, Servi D. et al. Incidence of adverse drug events and potential adverse drug events. Implications for prevention. ADE Prevention Study Group. JAMA 1995; Jul 5 274 (1) 29-34.
• 5 Committee on Quality of Health Care in America, Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. National Academies Press; 2001
• 6 Committee on Quality of Health Care in America, Institute of Medicine. To Err Is Human: Building a Safer Health System. Kohn LT, Corrigan JM, Donaldson MS. editors. The National Academies Press; 2000
• 7 Blumenthal D. Launching HITECH. N Engl J Med 2010; Feb 362 (5) 382-5.
• 8 Blumenthal D, Tavenner M. The "Meaningful Use" Regulation for Electronic Health Records. N Engl J Med 2010; Aug 363 (6) 501-4.
• 9 Pacific WHORO for the W Electronic health records: a manual for developing countries. WHO; 2006
• 10 Chaudhry B, Wang J, Wu S, Maglione M, Mojica W, Roth E. et al. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med 2006; May 16 144 (10) 742-52.
• 11 Bright TJ, Wong A, Dhurjati R, Bristow E, Bastian L, Coeytaux RR. et al. Effect of clinical decision-support systems: a systematic review. Ann Intern Med 2012; Jul 3 157 (1) 29-43.
• 12 Stead WW, Lin H. Computational technology for effective health care: immediate steps and strategic directions [Internet]. National Academy Press; 2009. [cited 2012 Dec 16] Available from: http://books.google.com/books?hl=en&lr=&id=HUAX-QzIPxV8C&oi=fnd&pg=PT1&dq=stead+lin+-computational+technology&ots=Sh6SFpD3L-c&sig=yfcG1YWKWdrHrJT9fqa7z5EeGC0
• 13 Buntin MB, Burke MF, Hoaglin MC, Blumenthal D. The Benefits Of Health Information Technology: A Review Of The Recent Literature Shows Predominantly Positive Results. Health Aff. 2011; Mar 1 30 (3) 464-71.
• 14 Fleming NS, Culler SD, McCorkle R, Becker ER, Ballard DJ. The Financial And Nonfinancial Costs Of Implementing Electronic Health Records In Primary Care Practices. Health Aff 2011; Mar 1 30 (3) 481-9.
• 15 Feblowitz JC, Wright A, Singh H, Samal L, Sittig DF. Summarization of clinical information: a conceptual model. J Biomed Inform 2011; Aug 44 (4) 688-99.
• 16 Laxmisan A, McCoy AB, Wright A, Sittig DF. Clinical Summarization Capabilities of Commercially-available and Internally-developed Electronic Health Records. Appl Clin Inform 2012; Feb 22 3 (1) 80-93.
• 17 Kwan JL, Lo L, Sampson M, Shojania KG. Medication Reconciliation During Transitions of Care as a Patient Safety StrategyA Systematic Review. Ann Intern Med 2013; Mar 5 158 (5_Part_2) 397-403.
• 18 Veer AJ de, Fleuren MA, Bekkema N, Francke AL. Successful implementation of new technologies in nursing care: a questionnaire survey of nurse-users. BMC Med Inform Decis Mak 2011; Oct 27 11 (1) 67.
• 19 Committee on Patient Safety and Health Information Technology, Institute of Medicine. Health IT and Patient Safety: Building Safer Systems for Better Care. The National Academies Press; 2012
• 20 McCoy AB, Waitman LR, Lewis JB, Wright JA, Choma DP, Miller RA. et al. A framework for evaluating the appropriateness of clinical decision support alerts and responses. J Am Med Inform Assoc 2012; May 1 19 (3) 346-52.
• 21 Vest JR, Gamm LD. Health information exchange: persistent challenges and new strategies. J Am Med Inform Assoc 2010; May 1 17 (3) 288-94.
• 22 Williams C, Mostashari F, Mertz K, Hogin E, Atwal P. From the Office of the National Coordinator: the strategy for advancing the exchange of health information. Health Aff (Millwood) 2012; Mar 31 (3) 527-36.
• 23 Sittig DF, Joe JC. Toward a statewide health information technology center (abbreviated version). South Med J 2010; Nov 103 (11) 1111-4.
• 24 Kuperman GJ. Health-information exchange: why are we doing it, and what are we doing?. J Am Med Inform Assoc 2011; Oct 18 (5) 678-82.
• 25 Hebel E, Middleton B, Shubina M, Turchin A. Bridging the chasm: effect of health information exchange on volume of laboratory testing. Arch Intern Med 2012; Mar 26 172 (6) 517-9.
• 26 Bailey JE, Wan JY, Mabry LM, Landy SH, Pope RA, Waters TM. et al. Does Health Information Exchange Reduce Unnecessary Neuroimaging and Improve Quality of Headache Care in the Emergency Department?. J Gen Intern Med 2013; Feb 28 (2) 176-83.
• 27 Frisse ME, Johnson KB, Nian H, Davison CL, Gadd CS, Unertl KM. et al. The financial impact of health information exchange on emergency department care. J Am Med Inform Assoc 2012; Jun 19 (3) 328-33.
• 28 Jones SS, Friedberg MW, Schneider EC. Health information exchange, Health Information Technology use, and hospital readmission rates. AMIA Annu Symp Proc 2011; 2011: 644-53.
• 29 Hincapie AL, Warholak TL, Murcko AC, Slack M, Malone DC. Physicians' opinions of a health information exchange. J Am Med Inform Assoc 2011; Feb 18 (1) 60-5.
• 30 Wright A, Soran C, Jenter CA, Volk LA, Bates DW, Simon SR. Physician attitudes toward health information exchange: results of a statewide survey. J Am Med Inform Assoc 2010; Feb 17 (1) 66-70.
• 31 Adler-Milstein J, Bates DW, Jha AK. A survey of health information exchange organizations in the United States: implications for meaningful use. Ann Intern Med 2011; May 17 154 (10) 666-71.
• 32 Adler-Milstein J, McAfee AP, Bates DW, Jha AK. The state of regional health information organizations: current activities and financing. Health Alf (Millwood) 2008; Feb 27 (1) w60-69.
• 33 Yasnoff SL WA. PUtting health it on the path to success. JAMA 2013; Mar 13 309 (10) 989-90.
• 34 Jha AK, Doolan D, Grandt D, Scott T, Bates DW. The use of health information technology in seven nations. Int J Med Inform 2008; Dec 77 (12) 848-54.
• 35 Sittig DF. Personal health records on the internet: a snapshot of the pioneers at the end of the 20th Century. Int J Med Inform 2002; Apr 65 (1) 1-6.
• 36 Google Health [Internet]. [cited 2012 Dec 3] Available from: http://www.google.com/intl/en_us/health/faq.html
• 37 Goldzweig CLCL, Towfigh AAAA, Paige NMNM, Orshansky GG, Haggstrom DADA, Beroes JMJM. et al. Systematic Review: Secure Messaging Between Providers and Patients, and Patients' Access to Their Own Medical Record: Evidence on Health Outcomes, Satisfaction, Efficiency and Attitudes. Washington (DC): Department ofVeterans Affairs (US); 2012
• 38 Zhou YY, Kanter MH, Wang JJ, Garrido T. Improved quality at Kaiser Permanente through e-mail between physicians and patients. Health Aff (Millwood) 2010; Jul 29 (7) 1370-5.
• 39 Simon GE, Ralston JD, Savarino J, Pabiniak C, Wentzel C, Operskalski BH. Randomized trial of depression follow-up care by online messaging. J Gen Intern Med 2011; Jul 26 (7) 698-704.
• 40 Green BB, Cook AJ, Ralston JD, Fishman PA, Catz SL, Carlson J. et al. Effectiveness of home blood pressure monitoring, Web communication, and pharmacist care on hypertension control: a randomized controlled trial. JAMA 2008; Jun 25 299 (24) 2857-67.
• 41 Leong SL, Gingrich D, Lewis PR, Mauger DT, George JH. Enhancing doctor-patient communication using email: a pilot study. J Am Board Fam Pract 2005; Jun 18 (3) 180-8.
• 42 Palen TE, Ross C, Powers JD, Xu S. Association of online patient access to clinicians and medical records with use of clinical services. JAMA 2012; Nov 21 308 (19) 2012-9.
• 43 Davis Giardina T, Singh H. Should patients get direct access to their laboratory test results? An answer with many questions. JAMA 2011; Dec 14 306 (22) 2502-3.
• 44 Ammenwerth E, Schnell-Inderst P, Hoerbst A. The Impact of Electronic Patient Portals on Patient Care: A Systematic Review of Controlled Trials. J Med Internet Res [Internet]. 2012 Nov 26 [cited 2012 Dec 14] 14. (6) Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3510722/
• 45 Wu W, Dasgupta S, Ramirez EE, Peterson C, Norman GJ. Classification accuracies of physical activities using smartphone motion sensors. J Med Internet Res 2012; 14 (5) e130.
• 46 Burns MN, Begale M, Duffecy J, Gergle D, Karr CJ, Giangrande E. et al. Harnessing context sensing to develop a mobile intervention for depression. J Med Internet Res 2011; 13 (3) e55.
• 47 Daugherty BL, Schap TE, Ettienne-Gittens R, Zhu FM, Bosch M, Delp EJ. et al. Novel technologies for assessing dietary intake: evaluating the usability of a mobile telephone food record among adults and adolescents. J Med Internet Res 2012; 14 (2) e58.
• 48 Kennedy CM, Powell J, Payne TH, Ainsworth J, Boyd A, Buchan I. Active assistance technology for health-related behavior change: an interdisciplinary review. J Med Internet Res 2012; 14 (3) e80.
• 49 Fanning J, Mullen SP, McAuley E. Increasing physical activity with mobile devices: a meta-analysis. J Med Internet Res 2012; 14 (6) e161.
• 50 Lau PWC, Lau EY, Wong DP, Ransdell L. A systematic review of information and communication technology-based interventions for promoting physical activity behavior change in children and adolescents. J Med Internet Res 2011; 13 (3) e48.
• 51 West JH, Hall PC, Hanson CL, Barnes MD, Giraud-Carrier C, Barrett J. There's an app for that: content analysis of paid health and fitness apps. J Med Internet Res 2012; 14 (3) e72.
• 52 Murray E. Web-Based Interventions for Behavior Change and Self-Management: Potential, Pitfalls, and Progress. Medicine 2.0. 2012; 1 (2) e3.
• 53 Webb TL, Joseph J, Yardley L, Michie S. Using the internet to promote health behavior change: a systematic review and meta-analysis of the impact of theoretical basis, use of behavior change techniques, and mode of delivery on efficacy. J Med Internet Res 2010; 12 (1) e4.
• 54 Eysenbach G. The law of attrition. J Med Internet Res 2005; 7 (1) e11.
• 55 Kelders SM, Kok RN, Ossebaard HC, Van Gemert-Pijnen JE. Persuasive system design does matter: a systematic review of adherence to web-based interventions. J Med Internet Res 2012; 14 (6) e152.
• 56 Lehto T, Oinas-Kukkonen H. Persuasive features in web-based alcohol and smoking interventions: a systematic review of the literature. J Med Internet Res 2011; 13 (3) e46.
• 57 Cugelman B, Thelwall M, Dawes P. Online interventions for social marketing health behavior change campaigns: a meta-analysis of psychological architectures and adherence factors. J Med Internet Res 2011; 13 (1) e17.
• 58 Brouwer W, Kroeze W, Crutzen R, De Nooijer J, De Vries NK, Brug J. et al. Which intervention characteristics are related to more exposure to internet-delivered healthy lifestyle promotion interventions? A systematic review. J Med Internet Res 2011; 13 (1) e2.
• 59 Morrison LG, Yardley L, Powell J, Michie S. What design features are used in effective e-health interventions? A review using techniques from Critical Interpretive Synthesis. Telemed J E Health 2012; Mar 18 (2) 137-44.
• 60 Eysenbach G. CONSORT-EHEALTH: improving and standardizing evaluation reports of Web-based and mobile health interventions. J Med Internet Res 2011; 13 (4) e126.
• 61 Nakamura MM, Ferris TG, DesRoches CM, Jha AK. Electronic health record adoption by children's hospitals in the United States. Arch Pediatr Adolesc Med 2010; Dec 164 (12) 1145-51.
• 62 Spooner SA. Special requirements of electronic health record systems in pediatrics. Pediatrics 2007; Mar 119 (3) 631-7.
• 63 Shifffman RN, Spooner SA, Kwiatkowski K, Brennan PF. Information technology for children's health and health care: report on the Information Technology in Children's Health Care Expert Meeting. September 21-22. 2000. J Am Med Inform Assoc 2001; Dec 8 (6) 546-51.
• 64 Hinman AR, Davidson AJ. Linking children's health information systems: clinical care, public health, emergency medical systems, and schools. Pediatrics 2009; Jan 123 Suppl 2 S67-73.
• 65 Rose AF, Schnipper JL, Park ER, Poon EG, Li Q, Middleton B. Using qualitative studies to improve the usability of an EMR. J Biomed Inform 2005; Feb 38 (1) 51-60.
• 66 Khajouei R, Jaspers MWM. CPOE system design aspects and their qualitative effect on usability. Stud Health Technol Inform 2008; 136: 309-14.
• 67 Saleem JJ, Patterson ES, Militello L, Anders S, Falciglia M, Wissman JA. et al. Impact of clinical reminder redesign on learnability, efficiency, usability, and workload for ambulatory clinic nurses. J Am Med Inform Assoc 2007; Oct 14 (5) 632-40.
• 68 Poissant L, Pereira J, Tamblyn R, Kawasumi Y. The impact of electronic health records on time efficiency of physicians and nurses: a systematic review. J Am Med Inform Assoc 2005; Oct 12 (5) 505-16.
• 69 Strom BL, Schinnar R, Aberra F, Bilker W, Hennessy S, Leonard CE. et al. Unintended effects of a computerized physician order entry nearly hard-stop alert to prevent a drug interaction: a randomized controlled trial. Arch Intern Med 2010; Sep 27 170 (17) 1578-83.
• 70 Baysari MT, Reckmann MH, Li L, Day RO, Westbrook JI. Failure to utilize functions of an electronic prescribing system and the subsequent generation of “technically preventable” computerized alerts. J Am Med Inform Assoc 2012; Nov 1 19 (6) 1003-10.
• 71 Han YY, Carcillo JA, Venkataraman ST, Clark RSB, Watson RS, Nguyen TC. et al. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system. Pediatrics 2005; Dec 116 (6) 1506-12.
• 72 Sittig DF, Ash JS, Zhang J, Osheroff JA, Shabot MM. Lessons from "Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system". Pediatrics 2006; Aug 118 (2) 797-801.
• 73 Longhurst CA, Parast L, Sandborg CI, Widen E, Sullivan J, Hahn JS. et al. Decrease in hospital-wide mortality rate after implementation of a commercially sold computerized physician order entry system. Pediatrics 2010; Jul 126 (1) 14-21.
• 74 Hales J, Ash JS, Lehmann CU, Longhurst CA, Spooner SA. Pediatric safety risks from unintended consequences of the use of EMRs designed for adults. AMIA Annu Symp Proc 2012; 1512.
• 75 Lowry SZ, Quinn MT, Ramaiah M, Schumacher RM, Patterson ES, North R. et al. Technical Evaluation, Testing, and Validation of the Usability of Electronic Health Records. National Institute of Standards and Technology. 2012
• 76 Schumacher R, Lowry S. Customized common industry format template for electronic health record usability testing. National Institute of Standards and Technology NISTIR; 2010. p. 7742.
• 77 Zhang J, Walji MF. TURF: toward a unified framework of EHR usability. J Biomed Inform 2011; Dec 44 (6) 1056-67.
• 78 Singh H, Ash JS, Sittig DF. Safety Assurance Factors for Electronic Health Record Resilience (SAFER): study protocol. BMC Med Inform Decis Mak 2013; Apr 12 13: 46.
• 79 Lowry SZ, Quinn MT, Ramaiah MD, Brick ES, Zhang Patterson J.. et al. A Human Factors Guide to Enhance EHR Usability of Critical User Interactions when Supporting Pediatric Patient Care. National Institute of Standards and Technology; 2012
• 80 Britto MT, Jimison HB, Munafo JK, Wissman J, Rogers ML, Hersh W. Usability testing finds problems for novice users of pediatric portals. J Am Med Inform Assoc 2009; 0ct 16 (5) 660-9.
• 81 Chun YJ, Patterson PE. A usability gap between older adults and younger adults on interface design of an Internet-based telemedicine system. Work 2012; Jan 1 41 (0) 349-52.
• 82 Shea S, Hripcsak G. Accelerating the use of electronic health records in physician practices. N Engl J Med 2010; Jan 21 362 (3) 192-5.
• 83 FASTSTATS — Homepage [Internet]. [cited 2012 Dec 14] Available from: http://www.cdc.gov/nchs/fastats/
• 84 Warner HR, Toronto AF, Veasey LG, Stephenson R. A mathematical approach to medical diagnosis: application to congenital heart disease.1961. MD Comput 1992; Feb 9 (1) 43-50.
• 85 Jensen PB, Jensen LJ, Brunak S. Mining electronic health records: towards better research applications and clinical care. Nat Rev Genet 2012; Jun 13 (6) 395-405.
• 86 Iavindrasana J, Cohen G, Depeursinge A, Müller H, Meyer R, Geissbuhler A. Clinical data mining: a review. Yearb Med Inform 2009; 121-33.
• 87 Milley A. Healthcare and data mining. Health Manag Technol 2000; 21 (8) 44-7.
• 88 Silver M, Sakata T, Su HC, Herman C, Dolins SB, O'Shea MJ. Case study: how to apply data mining techniques in a healthcare data warehouse. J Healthc Inf Manag 2001; 15 (2) 155-64.
• 89 Roque FS, Jensen PB, Schmock H, Dalgaard M, Andreatta M, Hansen T. et al. Using electronic patient records to discover disease correlations and stratify patient cohorts. PLoS Comput Biol 2011; 7 (8) e1002141.
• 90 Haerian K, Varn D, Vaidya S, Ena L, Chase HS, Friedman C. Detection of pharmacovigilance-re- lated adverse events using electronic health records and automated methods. Clin Pharmacol Ther 2012; Aug 92 (2) 228-34.
• 91 Coloma PM, Schuemie MJ, Trifiro G, Gini R, Herings R, Hippisley-Cox J. et al. Combining electronic healthcare databases in Europe to allow for large-scale drug safety monitoring: the EU- ADR Project. Pharmacoepidemiol Drug Saf 2011; Jan 20 (1) 1-11.
• 92 Jiang X, Boxwala AA, El-Kareh R, Kim J, Ohno-Machado L. A patient-driven adaptive prediction technique to improve personalized risk estimation for clinical decision support. Journal of the American Medical Informatics Association: J Am Med Inform Assoc 2012; Jun 1 19 (e1) e137-e144.
• 93 Himes BE, Dai Y, Kohane IS, Weiss ST, Ramoni MF. Prediction of chronic obstructive pulmonary disease (COPD) in asthma patients using electronic medical records. J Am Med Inform Assoc 2009; Jun 16 (3) 371-9.
• 94 Sarin P, Urman RD, Ohno-Machado L. An improved model for predicting postoperative nausea and vomiting in ambulatory surgery patients using physician-modifiable risk factors. J Am Med Inform Assoc 2012; Nov 1 19 (6) 995-1002.
• 95 Description — Heritage Health Prize [Internet]. [cited 2012 Dec 3] Available from: http://www.heritagehealthprize.com/c/hhp
• 96 Fialho AS, Cismondi F, Vieira SM, Reti SR, Sousa JMC, Finkelstein SN. Data mining using clinical physiology at discharge to predict ICU readmissions. Expert Syst Appl 2012; Dec 39 (18) 13158-65.
• 97 Meystre S. Electronic patient records: some answers to the data representation and reuse challenges. Findings from the section on Patient Records. Yearb Med Inform 2007; 47-9.
• 98 Kim H-E, Jiang X, Kim J, Ohno-Machado L. Trends in biomedical informatics: most cited topics from recent years. J Am Med Inform Assoc 2011; Dec 16 18 (Suppl 1) i166-i170.
• 99 Chapman WW, Nadkarni PM, Hirschman L, D'Avolio LW, Savova GK, Uzuner O. Overcoming barriers to NLP for clinical text: the role of shared tasks and the need for additional creative solutions. J Am Med Inform Assoc 2011; Oct 18 (5) 540-3.
• 100 Murff HJ, FitzHenry F, Matheny ME, Gentry N, Kotter KL, Crimin K. et al. Automated identification of postoperative complications within an electronic medical record using natural language processing. JAMA 2011; Aug 24 306 (8) 848-55.
• 101 Elkin PL, Froehling DA, Wahner-Roedler DL, Brown SH, Bailey KR. Comparison of natural language processing biosurveillance methods for identifying influenza from encounter notes. Ann Intern Med 2012; Jan 3 156 (1 Pt 1) 11-8.
• 102 Denny JC, Crawford DC, Ritchie MD, Bielinski SJ, Basford MA, Bradford Y. et al. Variants near FOXE 1 are associated with hypothyroidism and other thyroid conditions: using electronic medical records for genome- and phenome-wide studies. Am J Hum Genet 2011; Oct 7 89 (4) 529-42.
• 103 Xu H, Jiang M, Oetjens M, Bowton EA, Ramirez AH, Jeff JM. et al. Facilitating pharmacoge- netic studies using electronic health records and natural-language processing: a case study of warfarin. J Am Med Inform Assoc 2011; Aug 18 (4) 387-91.
• 104 Demner-Fushman D, Chapman WW, McDonald CJ. What can natural language processing do for clinical decision support?. J Biomed Inform 2009; 0ct 42 (5) 760-72.
• 105 Ardagna CA, De Capitani di Vimercati S, Foresti S, Grandison TW, Jajodia S, Samarati P. Access control for smarter healthcare using policy spaces. Computers & Security 2010; Nov 29 (8) 848-58.
• 106 Blobel B. Ontology driven health information systems architectures enable pHealth for empowered patients. Int J Med Inform 2011; Feb 80 (2) e17-25.
• 107 Heinze O, Birkle M, Koster L, Bergh B. Architecture of a consent management suite and integration into IHE-based Regional Health Information Networks. BMC Med Inform Decis Mak 2011; 11: 58.
• 108 Haas S, Wohlgemuth S, Echizen I, Sonehara N, Müller G. Aspects of privacy for electronic health records. Int J Med Inform 2011; Feb 80 (2) e26-31.
• 109 El Emam K, Moreau K, Jonker E. How Strong are Passwords Used to Protect Personal Health Information in Clinical Trials?. J Med Internet Res 2011; Feb 11 13 (1) e18.
• 110 Breaches Affecting 500 or More Individuals [Internet]. Office for Civil Rights, U.S. Dept. of Health and Human Services. Available from: http://www.hhs.gov/ocr/privacy/hipaa/adminis-trative/breachnotificationrule/breachtool.html
• 111 Romanosky S, Hoffman D, Acquisti A. Empirical Analysis of Data Breach Litigation. Temple University Legal Studies Research Paper; 2012. p. 30.
• 112 Boxwala AA, Kim J, Grillo JM, Ohno-Machado L. Using statistical and machine learning to help institutions detect suspicious access to electronic health records. J Am Med Inform Assoc 2011; Aug 18 (4) 498-505.
• 113 Fabbri D, LeFevre K, Hanauer DA. Explaining accesses to electronic health records. Proceedings of the 2011 workshop on Data mining for medicine and healthcare [Internet]. 2011 [cited 2012 Dec 3] p. 10-7. Available from: http://dl.acm.org/citation.cfm?id=2023585
• 114 Chen Y, Nyemba S, Malin BA. Auditing medical record accesses via healthcare interaction networks. AMIA Annu Symp Proc 2012; 93-102.
• 115 King JT, Smith B, Williams L. Modifying without a trace: general audit guidelines are inadequate for open-source electronic health record audit mechanisms. Proceedings of the 2nd ACM SIGHIT symposium on International health informatics [Internet]. 2012 [cited 2012 Dec 3] p. 305-14. Available from: http://dl.acm.org/citation.cfm?id=2110399
• 116 Murphy SN, Gainer V, Mendis M, Churchill S, Kohane I. Strategies for maintaining patient privacy in i2b2. J Am Med Inform Assoc 2011; Dec 18 Suppl 1 i103-108.
• 117 Benton A, Hill S, Ungar L, Chung A, Leonard C, Freeman C. et al. A system for de-identifying edical message board text. BMC Bioinformat- ics 2011; 12 Suppl 3 S2.
• 118 Meystre SM, Friedlin FJ, South BR, Shen S, Samore MH. Automatic de-identification of textual documents in the electronic health record: a review of recent research. BMC Med Res Methodol 2010; 10: 70.
• 119 Tu K, Klein-Geltink J, Mitiku TF, Mihai C, Martin J. De-identification ofprimary care electronic medical records free-text data in Ontario, Canada. BMC Med Inform Decis Mak 2010; 10: 35.
• 120 Dalianis H, Velupillai S. De-identifying Swedish clinical text — refinement of a gold standard and experiments with Conditional random fields. J Biomed Semantics 2010; 1 (1) 6.
• 121 Aberdeen J, Bayer S, Yeniterzi R, Wellner B, Clark C, Hanauer D. et al. The MITRE Identification Scrubber Toolkit: design, training, and assessment. Int J Med Inform 2010; Dec 79 (12) 849-59.
• 122 Gardner J, Xiong L. An integrated framework for de-identifying unstructured medical data. Data Knowl Eng 2009; 68 (12) 1441-51.
• 123 González DR, Carpenter T, Van Hemert JI, Wardlaw J. An open source toolkit for medical imaging de-identification. Eur Radiol 2010; Aug 20 (8) 1896-904.
• 124 El Emam K, Arbuckle L, Koru G, Eze B, Gaudette L, Neri E. et al. De-identification methods for open health data: the case of the Heritage Health Prize claims dataset. J Med Internet Res 2012; 14 (1) e33.
• 125 Loukides G, Gkoulalas-Divanis A, Malin B. Anonymization of electronic medical records for validating genome-wide association studies. Proc Natl Acad Sci U S A 2010; 107 (17) 7898-903.
• 126 O'Keefe CM, Connolly CJ. Privacy and the use of health data for research. Med J Aust 2010; Nov 1 193 (9) 537-41.
• 127 Rothstein MA. Is deidentification sufficient to protect health privacy in research?. Am J Bioeth 2010; Sep 10 (9) 3-11.
• 128 Benitez K, Malin B. Evaluating re-identification risks with respect to the HIPAA privacy rule. J Am Med Inform Assoc 2010; Apr 17 (2) 169-77.
• 129 El Emam K, Jonker E, Arbuckle L, Malin B. A systematic review of re-identification attacks on health data. PLoS ONE 2011; 6 (12) e28071.
• 130 Brown JS, Holmes JH, Shah K, Hall K, Lazarus R, Platt R. Distributed health data networks: a practical and preferred approach to multi-institutional evaluations of comparative effectiveness, safety, and quality of care. Med Care 2010; Jun 48 (6 Suppl) S45-51.
• 131 Wu Y, Jiang X, Kim J, Ohno-Machado L. Grid Binary LOgistic REgression (GLORE): building shared models without sharing data. J Am Med Inform Assoc 2012; Oct 19 (5) 758-64.
• 132 Rogers EM. Diffusion of Innovations, 5th Edition. Simon and Schuster; 2003