Download PDF
Methods Inf Med 2015; 54(04): 308-318
DOI: 10.3414/ME14-02-0119
Original Articles
Schattauer GmbH

Discussion of “Evidence-based Health Informatics: How Do We Know What We Know?”

N. Al-Shorbaji
1   World Health Organization, Geneva, Switzerland
,
L. Hanmer
2   Burden of Disease Research Unit and WHO Collaborating Centre for the Family of International Classifications, South African Medical Research Council, Cape Town, South Africa
,
R. Hussein
3   Biomedical Informatics Center of Excellence, Information Technology Institute, Ministry of Communications and Information Technology, Giza, Egypt
,
F. Magrabi
4   Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, Sydney, Australia
,
A. Moen
5   Institute for health and society, University of Oslo, Oslo, Norway
,
L. A. Moura
6   AM eHealth, Sao Paulo, Brazil
,
H. -A. Park
7   College of Nursing and Systems Biomedical Informatics Research Center, Seoul National University, Seoul, Korea
,
P. Scott
8   School of Computing, University of Portsmouth, Portsmouth, United Kingdom
› Author Affiliations
Further Information

Publication History

21 July 2015

Publication Date:
22 January 2018 (online)

    Permissions and Reprints

Summary

This article is part of a For-Discussion-Section of Methods of Information in Medicine about the paper “Evidence-based Health Informatics: How Do We Know What We Know?” written by Elske Ammenwerth [1]. It is introduced by an editorial. This article contains the combined commentaries invited to independently comment on the Ammenwerth paper. In subsequent issues the discussion can continue through letters to the editor.

With these comments on the paper “Evidence-based Health Informatics: How do we know what we know?”, written by Elske Ammenwerth [1], the journal seeks to stimulate a broad discussion on the challenges of evaluating information processing and information technology in health care. An international group of experts has been invited by the editor of Methods to comment on this paper. Each of the invited commentaries forms one section of this paper.

Keywords

Medical informatics - evaluation studies - publication bias - meta-analysis - qualitative research - review literature - information science - evidence-based medicine
 

  • References

  • 1 Ammenwerth E. Evidence-based Health Informatics: How Do We Know What We Know?. Methods Inf Med 2015; 54 (04) 298-307.
    Article in Thieme ConnectPubMedGoogle Scholar
  • 2 World Health Organization. World Health assembly resolution WHA58.28. Geneva: World Health Organization; 2005. http://www.who.int/ehealth/about/en/ [access on 21 April 2015].
    Google Scholar
  • 3 World Health Organization. Proposed Programme Budget 2014/2015. Geneva: World Health Organization; 2013. http://www.who.int/about/resources_planning/A66_7-en.pdf?ua=1&ua=1 [access on 23 April 2015].
    Google Scholar
  • 4 World Health Organization. Proposed Programme Budget 2016/2017. Geneva: World Health Organization; 2013. http://www.who.int/about/resources_planning/EB136_34-en.pdf?ua=1 [access on 23 April 2015].
    Google Scholar
  • 5 World Health Organization. World Health assembly resolution WHA66.24. Geneva: World Health Organization; 2013. http://apps.who.int/gb/ebwha/pdf_files/WHA66/A66_R24-en.pdf?ua=1 [access on 21 April 2015].
    Google Scholar
  • 6 Call to action on global eHealth evaluation: consensus statement of the WHO Global eHealth Evaluation Meeting, Bellagio, September 2011. In: eHealth evaluation evidence. Bellagio: The Bellagio eHealth Evaluation Group; 2011. Available from http://www.ghdonline.org/uploads/The_Bellagio_eHealth_Evaluation_Call_to_Action-Release.docx [accessed 15 April 2015].
    Google Scholar
  • 7 Bulletin of the World Health Organization. Special theme issue: eHealth, Volume 90, Number 5, May. 2012: 321-400. http://www.who.int/bulletin/volumes/90/5/en/ [accessed 15 April 2015].
    Google Scholar
  • 8 Al-Shorbaji N, Geissbuhler A. Establishing an evidence base for e-health: the proof is in the pudding. Bulletin, of the World Health Organization. V. 90. 5 http://www.who.int/bulletin/volumes/90/5/12-106146.pdf?ua=1 [accessed 20 April 2015].
    PubMedGoogle Scholar
  • 9 Piette JD, Lun KC, Moura LA, Fraser HS, Mechael PN, Powell J, Khoja SR. Impacts of e-health on the outcomes of care in low- and middle-income countries: where do we go from here?. Bull World Health Organ 2012; 90 (05) 365-372. doi:10.2471/BLT.11.099069.
    CrossrefPubMedGoogle Scholar
  • 10 UMIT, EFMI WG Eval. Health IT Evaluation Database. 2014. [accessed 01 May 2015]; Available from http://evaldb.umit.at.
    Google Scholar
  • 11 Nguyen L, Bellucci E, Nguyen LT. Electronic health records implementation: An evaluation of information system impact and contingency factors. Int J Med Inform 2014; 83 (11) 779-796. doi:10.1016/j.ijmedinf.2014.06.011.
    CrossrefPubMedGoogle Scholar
  • 12 Moja L, Kwag KH, Lytras T, Bertizzolo L, Brandt L, Pecoraro V, et al. Effectiveness of computerized decision support systems linked to electronic health records: A systematic review and meta-analysis. A J Public Health 2014; 104 (12) e12-22.
    PubMedGoogle Scholar
  • 13 A new global partnership: eradicate poverty and transform economies through sustainable development. Report of the High-level Panel of Eminent Persons on the Post-2015 Development Agenda. New York: United Nations Publications; 2013. Available at http://www.un.org/sg/management/pdf/HLP_P2015_Report.pdf [accessed 9 April 2015].
    Google Scholar
  • 14 Hussein R. A Review of Realizing the Universal Health Coverage (UHC) Goals by 2030; Part 1-Status quo, Requirements, and Chal- lenges. J Med Syst 2015; 39 (07) 254 doi: 10.1007/s10916–015–0254-y.
    PubMedGoogle Scholar
  • 15 World Health Organization and World Bank Group. Monitoring progress towards universal health coverage at country and global levels: framework, measures and targets. Geneva: WHO; 2014. Available at http://www.who.int/healthinfo/universal_health_coverage/en/ [accessed 9 April 2015].
    Google Scholar
  • 16 Hussein R. A Review of Realizing the Universal Health Coverage (UHC) Goals by 2030; Part 2-What is the Role of eHealth and Technology?. J Med Syst 2015; 39 (07) 255 doi: 10.1007/s10916–015–0255-x.
    PubMedGoogle Scholar
  • 17 Mechael P. Evaluation of IDRC-supported eHealth projects: final report. 2011. [cited 2015 Apr 10]; Available from http://idl-bnc.idrc.ca/dspace/handle/10625/46411.
    Google Scholar
  • 18 Sinha C, Garro-Strauss D. Research on eHealth Across Health Systems: Contributions to Strengthen a Field. In Elder L, Emdon H, Fuchs R, Petrazzini B. editors Connecting ICTs to Development the IDRC Experience. Ottawa: Anthem Press; 2013. Available at http://idl-bnc.idrc.ca/dspace/bitstream/10625/52228/1/IDL-52228.pdf [accessed 9 April 2015].
    Google Scholar
  • 19 Labrique AB, Vasudevan L, Kochi E, Fabricant R, Mehl G. mHealth innovations as health system strengthening tools: 12 common applications and a visual framework. Glob Health Sci Pract 2013; 1 (02) 160-171.
    CrossrefPubMedGoogle Scholar
  • 20 Mehl G, Labrique A. Prioritizing integrated mHealth strategies for universal health coverage. Science 2014; 345 6202 1284-1287.
    CrossrefPubMedGoogle Scholar
  • 21 Mehl G, Vasudevan L, Gonsalves L, Berg M, Seimon T, Temmerman M, Labrique A. Harnessing mHealth in low-resource settings to overcome health system constraints and achieve universal access to health. In Marsch LA, Lord SE, Dallery J. editors Behavioral Healthcare and Technology: Using Science-Based Innovations to Transform Practice. Oxford University Press; 2014: 369.
    Google Scholar
  • 22 Walshe K, McKee M, McCarthy M, Groenewegen P, Hansen J, Figueras J. et al. Health systems and policy research in Europe: Horizon 2020. Lancet 2013; 382 9893 668-669.
    CrossrefPubMedGoogle Scholar
  • 23 Ammenwerth E, Schnell-Inderst P, Siebert U. Vision and challenges of Evidence-Based Health Informatics: a case study of a CPOE meta-analysis. Int J Med Inform 2010; 79 (04) e83-88.
    CrossrefPubMedGoogle Scholar
  • 24 Rigby M, Ammenwerth E, Beuscart-Zephir MC, Brender J, Hypponen H, Melia S. et al. Evidence Based Health Informatics: 10 Years of Efforts to Promote the Principle. Joint Contribution of IMIA WG EVAL and EFMI WG EVAL. Yearb Med Inform 2013; 8 (01) 34-46.
    Article in Thieme ConnectPubMedGoogle Scholar
  • 25 Coiera E. The guide to health informatics. third edition. CRC Press; 2015
    Google Scholar
  • 26 Darking M, Anson R, Bravo F, Davis J, Flowers S, Gillingham E. et al. Practice-centred evaluation and the privileging of care in health information technology evaluation. BMC Health Serv Res 2014; 14: 243.
    CrossrefPubMedGoogle Scholar
  • 27 Eisenstein EL, Juzwishin D, Kushniruk AW, Nahm M. Defining a framework for health information technology evaluation. Stud Health Technol Inform 2011; 164: 94-99.
    PubMedGoogle Scholar
  • 28 Yusof MM, Papazafeiropoulou A, Paul RJ, Stergioulas LK. Investigating evaluation frameworks for health information systems. Int J Med Inform 2008; 77 (06) 377-385.
    CrossrefPubMedGoogle Scholar
  • 29 Moen A, Knudsen LMM. Nursing Informatics - decades of contributions to health informatics. Healthc Inform Res 2013; 19 (02) 86-92.
    CrossrefPubMedGoogle Scholar
  • 30 Moen A. Nursing Leadership when an Electronic Patient Record System is Introduced in Norwegian Hospitals [Doctoral Dissertation]. Oslo: University of Oslo; 2002
    Google Scholar
  • 31 Greenhalgh T. Outside the Box: Why are Cochrane reviews so boring?. Br J Gen Pract 2012; 62 (600) 371.
    CrossrefPubMedGoogle Scholar
  • 32 Koppel R, Metlay JP, Cohen A, Abaluck B, Localio AR, Kimmel SE. et al. Role of Computerized Physician Order Entry Systems in Facilitating Medication Errors. JAMA 2005; 293 (10) 1197-1203.
    CrossrefPubMedGoogle Scholar
  • 33 Nes S, Moen A. Constructing standards - A study of nurses negotiating with multiple modes of knowledge. Journal of Workplace Learning 2010; 22 (06) 376-393.
    CrossrefPubMedGoogle Scholar
  • 34 Coiera E, Aarts J, Kulikowski C. The dangerous decade. J Am Med Inform Assoc 2012; 19 (01) 2-5.
    CrossrefPubMedGoogle Scholar
  • 35 Hurlen P, Borthne A, Dahl FA, Ostbye T, Gulbrandsen P. Does PACS improve diagnostic accuracy in chest radiograph interpretations in clinical practice?. Eur J Radiol 2012; 81 (01) 173-7.
    CrossrefPubMedGoogle Scholar
  • 36 WHO-ITU National eHealth Toolkit. http://www.who.int/ehealth/publications/overview.pdf?ua=1&ua=1 2012
    PubMedGoogle Scholar
  • 37 Keeley L, Doblin Inc IBF. Venturing & Partnering, February. 2011 http://corporateventuringconference.com/2012/PRESENTATIONS/2011/04LarryKeeley.pdf.
    PubMedGoogle Scholar
  • 38 Winter A, Haux R, Ammenwerth E, Brigl B, Hellrung N, Jahn F. Health Information Systems: Architectures and Strategies. 2nd ed. London: Springer; 2011
    Google Scholar
  • 39 AHRQ. Health IT Evaluation Toolkit and Evaluation Measures Quick Reference Guides. Available from. http://healthit.ahrq.gov/health-it-tools-and-resources/health-it-evaluation-toolkit-and-evaluation-measures-quick-reference.
    PubMedGoogle Scholar
  • 40 Congressional Budget Office. Evidence on the Costs and Benefits of Health Information Technology, May 2008. Available from. https://www.cbo.gov/sites/default/files/05-20-healthit.pdf.
    PubMedGoogle Scholar
  • 41 Scott PJ. Exploiting the information revolution:call for independent evaluation of the latest English national experiment. J Innov Health Inform 2015; 22 (01) 244-249.
    PubMedGoogle Scholar
  • 42 Crump B. Should we use large scale healthcare interventions without clear evidence that benefits outweigh costs and harms? Yes. BMJ 2008; 336 7656 1276.
    CrossrefPubMedGoogle Scholar
  • 43 Landefeld CS, Shojania KG, Auerbach AD. Should we use large scale healthcare interventions without clear evidence that benefits outweigh costs and harms? No. BMJ 2008; 336 7656 1277.
    CrossrefPubMedGoogle Scholar
  • 44 Song PH, McAlearney AS, Robbins J, McCullough JS. Exploring the business case for ambulatory electronic health record system adoption. J Healthc Manag 2011; 56 (03) 169-180.
    PubMedGoogle Scholar
  • 45 Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ 2008; 337: a1655.
    PubMedGoogle Scholar
  • 46 Lilford RJ, Chilton PJ, Hemming K, Girling AJ, Taylor CA, Barach P. Evaluating policy and service interventions: framework to guide selection and interpretation of study end points. BMJ 2010; 341: c4413.
    CrossrefPubMedGoogle Scholar
  • 47 Liu JL, Wyatt JC. The case for randomized controlled trials to assess the impact of clinical information systems. J Am Med Inform Assoc 2011; 18 (02) 173-180.
    CrossrefPubMedGoogle Scholar
  • 48 Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W. et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ 2015; 350: h1258.
    CrossrefPubMedGoogle Scholar
  • 49 De Silva MJ, Breuer E, Lee L, Asher L, Chowdhary N, Lund C. et al. Theory of Change: a theory-driven approach to enhance the Medical Research Council’s framework for complex interventions. Trials 2014; 15: 267.
    CrossrefPubMedGoogle Scholar
  • 50 Nykänen P, Brender J, Talmon J, de Keizer N, Rigby M, Beuscart-Zephir MC. et al. Guideline for good evaluation practice in health informatics (GEP-HI). Int J Med Inform 2011; 80 (12) 815-827.
    CrossrefPubMedGoogle Scholar
  • 51 Friedman C, Wyatt J. Evaluation methods in biomedical informatics. 2nd ed. New York: Springer Verlag; 2006
    Google Scholar
  • 52 Friedman CP, Abbas UL. Is medical informatics a mature science? A review of measurement practice in outcome studies of clinical systems. Int J Med Inform 2003; 69 2–3 261-272.
    CrossrefPubMedGoogle Scholar
  • 53 UK Office of Government Commerce. Managing Successful Projects with PRINCE2. London: Office of Government Commerce; 2009
    Google Scholar
  • 54 Scott PJ, Briggs JS, Prytherch D. Health informatics - where’s the evidence?. HI Now. 2010. 5–1 10-12. Available from http://www.bcs.org/upload/pdf/hinow-autumn-2010.pdf.
    Google Scholar