Level of Evidence as a Future Gold Standard for the Content Quality of Health Resources on the Internet

Summary

Objective: An assessment of the quality of health information on the Internet is an absolute necessity. In this study ‘sensitive’ information was defined as information found in documents published on the Internet, which could be used in a medical decision. For sensitive information, the main criterion chosen for the quality of the information was an indication of the level of evidence. A survey was conducted using the CISMeF health catalogue to assess how often a score of the level of evidence is mentioned in the information accessible on the Internet in French-language health resources.

Methods: Since 1999, members of the CISMeF team have systematically been searching for all documents containing ‘sensitive’ information and verifying whether the level of evidence was explicitly indicated as a score at least once in the document.

Results: As of June 2001, 10,190 resources were included in CISMeF; including 2964 textual ‘sensitive’ resources (29.1%). Out of all these resources, only 4.7% (95% confidence interval: 4.0 - 5.5%) indicated the level of evidence. A statistically significant difference in the prevalence of indicating the level of evidence according to resource types (e.g., 18.1% for guidelines compared to 0.0% for teaching material), year of publication (almost three times greater in 1997-2001 compared with 1990-1996) and publishers was observed.

Conclusion: As the number of people accessing the growing amount of information on the Internet is increasing daily, publishers have an ethical obligation to inform their readers about the validity of ‘sensitive’ information their sites contain. However, the vast majority of the French language Internet resources that were surveyed do not mention a score of the level of evidence for their sensitive information.

Keywords

Internet - quality control - evidence-based medicine - level of evidence

• References

• 1 Code of ethics of the Internet Healthcare Coalition. Available from: URL: www.ihealthcoalition.org/ethics/ethics.html
• 2 Ambre J, Guard R, Perveiler FM, Renner J, Rippen H. Health Information Technology Institute. Working Draft White Paper: Criteria for Assessing the Quality of Health Information on the Internet. Available from: URL: http://hitiweb.mitretek.org/hswg&http://hitiweb.mitretek.org/docs/policy.pdf
• 3 Boyer C, Selby M, Scherrer JR, Appel RD. The Health On the Net Code of Conduct for medical and health Websites. Comput Biol Med 1998; 28: 603-10.
• 4 Eysenbach G, Diepgen T, Lampe K, Brickley D. EU-project medCERTAIN: Certification and Rating of Trustworthy and Assessed Health Information on the Net. Stud Health Technol Inform 2000; 77: 279-83.
• 5 Darmoni SJ, Leroux V, Daigne M, Thirion B, Santamaria P, Duvaux C. In: Albert A, Roger-France FH, Degoulet D, Fieschi M. eds. Critères de qualité de l’information de santé sur l’Internet. Santé et Réseaux Informatiques. Informatique et Santé Paris: Springer Verlag France; 1998: 162 74.: Available from: URL: www.hbroussais.fr/Broussais/InforMed/InforSante/Volume10/Vol%2010-pdf/10-19.pdf
• 6 Winker MA, Flanagin A, Chi-Lum B. et al. Guidelines for Medical and Health Information Sites on the Internet. Principles Governing AMA Web Sites. JAMA 2000; 283: 1600-6. Available from: URL: http://jama.ama-assn.org/issues/v283n12/pdf/jsc00054.pdf
• 7 Tatsumi H, Mitani H, Haruki Y, Ogushi Y. Internet medical usage in Japan: Current situation and issues. J Med Internet Res 2001; 3: e12 Available from: URL: www.jmir.org/2001/1/e12/
• 8 Impicciatore P, Pandolfini C, Casella N, Bonat M. Reliability of health information for the public on the world wide web: systematic survey of advice on managing fever in children at home. BMJ 1997; 314: 1875-8.
• 9 McClung HJ, Murray RD, Heitlinger LA. The Internet as a source for Current Patient Information. Pediatrics 1998; 101: e2. Available from: URL: www.pediatrics.org/cgi/content/full/101/6/e2
• 10 Sandvik H. Health information and interaction on the internet: a survey of female urinary incontinence. BMJ 1999; 319: 29-32.
• 11 Pandolfini C, Impicciatore P, Bonati M. Parents on the web: risks for quality management of cough in children. Pediatrics 2000; 105: e1.
• 12 Hernández-Borges AA, Macías-Cervi P, Gaspar-Guardado MA, Torres-Álvarez de Arcaya ML, Ruiz-Rabaza A, Jiménez-Sosa A. Can Examination of WWW Usage Statistics and other Indirect Quality Indicators Distinguish the Relative Quality of Medical Web Sites?. J Med Internet Res 1999; 1: e5. Available from: URL: www.jmir.org/1999/1/e1/index.htm
• 13 Griffiths KM, Christensen H. Quality of web based information on treatment of depression: cross sectional survey. BMJ 2000; 321: 1511-5.
• 14 Darmoni SJ, Haugh M, Lukacs B, Boissel JP. Level of evidence should be gold standard. BMJ 2001; 322: 1367.
• 15 Darmoni SJ, Leroy JP, Baudic F, Douyère M, Piot J, Thirion B. CISMeF: a structured health catalogue. Methods Inf Med 2000; 39: 30-5.
• 16 Joubert M, Aymard S, Fieschi D, Fieschi M. Quality Criteria and Access Characteristics of Web Sites: Proposal for the Design of a Health Internet Directory. Proc AMIA Symp. 1999: 824-8.
• 17 Kamel Boulos MN, Roudsari AV, Gordon C, Muir Gray JA. The Use of Quality Benchmarking in Assessing Web Resources for the Dermatology Virtual Branch Library of the National electronic Library for Health (NeLH). J Med Internet Res 2001; 3: e5. Available from: URL: www.jmir.org/2001/1/e5
• 18 Darmoni SJ, Thirion B, Leroy JP, Douyère M, Piot J. The Use of Dublin Core Metadata in a Structured Health Resource Guide on the Internet. Bull Med Libr Assoc. In press 2001
• 19 Weibel S, Juha H. DC-5:The Helsinki Metadata Workshop; A Report on the Workshop and Subsequent Developments. D-Lib Magazine. 1998 Available from: URL: www.dlib.org/dlib/february98/02weibel.html
• 20 Thirion B, Darmoni SJ. Simplified access to MeSH Tree Structures on CISMeF. Bull Med Libr Assoc 1999; 87: 480-1.
• 21 Woolf SH. Practice guidelines, a new reality in medicine; I. Recent developments. Arch Intern Med 1990; 150: 1811-8.
• 22 Bossard N, Boissel JP, Duru G. La hiérarchisation selon le niveau de preuve des données sources, avant leur intégration dans une synthèse, en matière d’efficacité thérapeutique. Therapie 1996; 51: 261-4.
• 23 Boissel JP, Bossard N, Chauvin F. et al. Niveau de preuve pour l’information thérapeutique. In: Bouvenot G. ed. L’information thérapeutique. Paris: Masson; 2000: 205-26.
• 24 Wayne WD. Biostatistics: a foundation for analysis in the health sciences. 5th edition. John Wiley & Sons; 1987
• 25 Statistical Packaged for the Social Science: Users guide. SPSS Inc; 2000
• 26 Darmoni SJ, Thirion B, Leroy JP. et al. A search tool based on ‘encapsulated’ MeSH thesaurus to retrieve quality health resources on the Internet. Med Inform Internet Med. In press 2001
• 27 Fervers B, Hardy J, Blanc-Vincent MP. et al. SOR: project methodology. Br J Cancer 2001; 84: 8-16.
• 28 Ray-Coquard I, Philip T, Lehmann M, Fervers B, Farsi F, Chauvin F. Impact of a clinical guidelines program for breast and colon cancer in a French cancer center. JAMA 1997; 278: 1591-5.
• 29 LeBeux P, LeDuff F, Fresnel A. et al. The French Virtual Medical University. Stud Health Technol Inform 2000; 77: 554-62.
• 30 Nony P, Cucherat M, Boissel JP. Implication of evidence-based medicine in prescription guidelines taught to French medical students: current status in the cardiovascular field. Clin Pharmacol Ther 1999; 66: 173-84.
• 31 Wyatt J. Commentary: Measuring quality and impact of the world wide web. BMJ 1997; 314: 1879-80. Available from: URL: www.bmj. com/archive/7098ip2.htm
• 32 Silberg WM, Lundberg GD, Musacchio RA. Assessing, Controlling, and Assuring the Qua lity of Medical Information on the Internet: Caveant Lector et Viewor – Let the Reader and Viewer Beware. JAMA 1997; 277: 1244-5.
• 33 Berland GK, Elliott MN, Morales LS. et al. Health information on the Internet: accessibility, quality, and readability in English and Spanish. JAMA 2001; 285: 2612-21.