Quality of the ICD-11 Beta Draft from the German Perspective: Evaluation Based on the Alphabet of ICD-10-GM 2017

Summary

Objectives: The German Association for Medical Informatics, Biometry and Epidemiology implemented a field test for the ICD-11 Beta Draft. Aim was to analyze completeness and appropriateness of the ICD-11 Beta Draft in its entire breadth.

Methods: Starting point was the synonym thesaurus (“Alphabet”) of the German modification of ICD-10. The Alphabet included a list of diagnoses terms that supports the coding of diagnoses with ICD-10. A sample of 60,328 diagnosis terms was drawn to be mapped to the ICD-11 Beta Draft. A subsample of 13,975 diagnosis terms was prepared for assessing reliability. First, the coders had to assign a diagnosis term from the sample to an appropriate English one. This included the automatic selection of the respective code from the ICD-11 Beta Draft. Secondly, the coders had to answer questions regarding completeness, appropriateness, and other issues.

Results: Finally, 49,184 results from 36 coders were available for the analysis. Problems with completeness were indicated in 4.7% of the results, problems with appropriateness in 5.3%. On the level of chapters, Cohen’s kappa reached grade “fair” at a maximum. The coders agreed in 31.4% of the terms.

Conclusions: Problems with the ICD-11 Beta Draft appeared to be moderate. Completeness was high, reliability was low as it is known for ICD-10. Concerns with the structure of the ICD-11 Beta Draft were noted, e. g. for neoplasms. A post processing of the ICD-11 Beta Draft seems to be sufficient with regard to the content. Methodologically, a thorough review of the structure might be advisable.

Keywords

Classification - coding - diagnoses - International Classification of Diseases - reliability

• References

• 1 Gersenovic M. The ICD family of classifications.. Methods Inf Med 1995; 34: 172-175.
• 2 Graubner B. ICD and OPS.. Historical development and current situation. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz 2007; 50: 932-943. German.
• 3 Jakob R, Üstün B, Madden R. et al. The WHO Family of International Classifications.. Bundesgesundheitsbl - Gesundheitsforsch - Gesundheitsschutz 2007; 50: 924-931.
• 4 Mansky T, Scriba PC, Fassl H. et al. Diagnosis encoding: how and to what purpose?. Dtsch Med Wochenschr 1986; 111: 1707-1708. German.
• 5 Coster CD, Quan H, Finlayson A. et al. Identifying priorities in methodological research using ICD9-CM and ICD-10 administrative data: report from an international consortium.. BMC Health Serv Res 2006; 6: 77.
• 6 Henderson T, Shepheard J, Sundararajan V. Quality of diagnosis and procedure coding in ICD-10 administrative data.. Med Care 2006; 44: 1011-1019.
• 7 World Health Organization Collaborating Centre for Classification, Terminology and Standards. ICD-11 Field Trial: Code-Recode ICD-11 Training Materials. May 2015
• 8 Evans SC, Roberts MC, Keeley JW. et al. Vignette methodologies for studying clinicians’ decisionmaking: Validity, utility, and application in ICD-11 field studies.. International Journal of Clinical and Health Psychology 2015; 15: 160-170.
• 9 Rodrigues JM, Robinson D, Della Mea V. et al. Semantic Alignment between ICD-11 and SNOMED CT.. Stud Health Technol Inform 2015; 216: 790-794.
• 10 Cardillo E, Eccher C, Serafini L. et al. Logical Analysis of Mappings between Medical Classification Systems. In Dochev D, Pistore M, Traverso P. editors. Artificial Intelligence: Methodology, Systems, and Applications. AIMSA 2008. Lecture Notes in Computer Science, Vol 5253. Berlin, Heidelberg: Springer; 2008
• 11 First MB. Harmonisation of ICD-11 and DSM-V: opportunities and challenges.. The British Journal of Psychiatry 2009; 195: 382-390.
• 12 Cornet R, de Keizer BF, Abu-Hanna A. A framework for characterizing terminological systems.. Methods Inf Med 2006; 45: 253-266.
• 13 Stausberg J, Dahmen B, Drösler SE. A methodological framework for the conversion of procedure classifications.. Methods Inf Med 2005; 44: 57-65.
• 14 Graubner B. ICD-10-SGBV and ICD-10-diagnosenthesaurus - advantages and disadvantages as well as further development.. Stud Health Technol Inform 2000; 77: 161-164.
• 15 World Health Organization. ICD-11 Reference Guide Draft. 2017-05-10.
• 16 Donada M, Kostanjsek N, Della Mea V. et al. Piloting a Collaborative Web-Based System for Testing ICD-11.. Stud Health Technol Inform 2017; 235: 466-470.
• 17 Landis JR, Koch GG. The measurement of observer agreement for categorical data.. Biometrics 1977; 33: 159-174.
• 18 Campbell JR, Carpenter P, Sneiderman C. et al. Phase II evaluation of clinical coding schemes: completeness, taxonomy, mapping, definitions, and clarity.. CPRI Work Group on Codes and Structures. J Am Med Inform Assoc 1997; 4: 238-251.
• 19 Koch H, Graubner B, Brenner G. Erprobung der Diagnosenverschlüsselung mit der ICD-10 in der Praxis des niedergelassenen Arztes. Köln: Deutscher Ärzte-Verlag; 1998
• 20 Stausberg J, Lehmann N, Kaczmarek D. et al. Reliability of diagnoses coding with ICD-10.. International Journal of Medical Informatics 2008; 77: 50-77.
• 21 Wockenfuss R, Frese T, Herrmann K. et al. Threeand four-digit ICD-10 is not a reliable classification system in primary care.. Scand J Prim Health Care 2009; 27: 131-136.
• 22 Nilsson G, Petersson H, Åhlfeld H. et al. Evaluation of three Swedish ICD-10 primary care versions: reliability and ease of use in diagnostic coding.. Methods Inf Med 2000; 39: 325-331.
• 23 Januel JM, Luthi JC, Quan H. et al. Improved accuracy of co-morbidity coding over time after the introduction of ICD-10 administrative data.. BMC Health Serv Res 2011; 11: 194.
• 24 Claus C, Popert U, Bösner S. et al. Wie viel Zeit kostet Kodierung objektiv - Boyd-Fallvignette zum Vergleich von AKR und CodA-Liste. Meeting abstract 2011. Available from: http://dx.doi.org/10.3205/11fom170.
• 25 Hennessy DA, Quan H, Faris PD. et al. Do coder characteristics influence validity of ICD-10 hospital discharge data?. BMC Health Serv Res 2010; 10: 99.
• 26 Stausberg J, Lang H, Obertacke U. et al. Classifications in Routine Use: Lessons from ICD-9 and ICPM in Surgical Practice.. J Am Med Inform Assoc 2001; 8: 92-100.
• 27 Elkin PL, Brown SH, Carter J. et al. Guideline and quality indicators for development, purchase and use of controlled health vocabularies.. Int J Med Inform 2002; 68: 175-186.
• 28 Southern DA, Pincus HA, Romano PS. et al; World Health Organization ICD-11 Revision Topic Advisory Group on Quality & Safety; World Health Organization ICD-11 Revision Topic Advisory Group on Quality & Safety. Enhanced capture of healthcare-related harms and injuries in the 11th revision of the International Classification of Diseases (ICD-11).. Int J Qual Health Care 2016; 28: 136-142.