Methods Inf Med 2007; 46(05): 558-566
DOI: 10.1160/ME0382
Paper
Schattauer GmbH

Comparing Properties of Audit Data and Routinely Collected Register Data in Case of Performance Assessment of Hip Fracture Treatment in Finland

R. Sund
1   National Research and Development Centre for Welfare and Health, Helsinki, Finland
,
I. Nurmi-Lüthje
2   Health Centre of Kouvola Region, Kouvola, Finland
,
P. Lüthje
3   Kuusankoski Regional Hospital, Kuusankoski, Finland
,
S. Tanninen
3   Kuusankoski Regional Hospital, Kuusankoski, Finland
,
A. Narinen
2   Health Centre of Kouvola Region, Kouvola, Finland
,
I. Keskimäki
1   National Research and Development Centre for Welfare and Health, Helsinki, Finland
› Author Affiliations
Further Information

Publication History

Publication Date:
22 January 2018 (online)

Summary

Objectives: To compare prospective audit data and secondary administrative register data in the production of performance assessment information in the case of hip fracture treatment, and to cross-validate the quality of information.

Methods: First, a conceptual model for the performance assessment of hip fracture treatment was defined. This model was then utilized in comparisons between the prospective audit data concerning 106 consecutive hip fracture patients from the Kuusankoski Regional Hospital and corresponding register data from the Finnish Health Care Register and the Causes of Death Register. We examined the completeness of registration of patients and also the accuracy and degree of completeness of the registered data. Observed differences were checked against the medical records.

Results: Register data lack clinical detail, but outperform prospective data in the recording of inpatient care history. Completeness of the register data is very good. The accuracy of easily measurable variables in the register is at least 95%. The agreement between register and audit data was 86.3% for detailed hip fracture diagnosis. Polyserial correlation between the functional dependency variables was 0.68.

Conclusions: Register and audit data have certain limitations and problems, but both seem to be suitable for the performance assessment of hip fracture treatment. To improve the feasibilityof the administrative register data, the voluntary input of additional hip fracture event data to the register should be made possible. Standardized instructions that guide the use of available register classifications in a sensible way would improvethe quality of data.

 
  • References

  • 1 Aday LA, Begley CE, Lairson DR, Slater CH. Evaluating the healthcare system: Effectiveness, efficiency, and equity 2nd edition.. Chicago: Health Administration Press; 1998
  • 2 OECD. Measuring up – Improving health system performance in OECD countries.. Paris: Organisation for Economic Co-operation and Development; 2002
  • 3 Hasman A, Safran C, Takeda H. Quality of health care: informatics foundations. Methods Inf Med 2003; 42 (05) 509-18.
  • 4 Ammenwerth E, Haux R, Kulikowski C, Bohne A, Brandner R, Brigl B. et al. Medical informatics and the quality ofhealth: new approaches to support patient care – findings from the IMIA Yearbook of Medical Informatics 2003. Methods Inf Med 2003; 42 (02) 185-9.
  • 5 Ammenwerth E, Ehlers F, Eichstadter R, Haux R, Pohl U, Resch F. Systems analysis in health care: framework and example. Methods Inf Med 2002; 41 (02) 134-40.
  • 6 Stefanelli M. Knowledge management to support performance-based medicine. Methods Inf Med 2002; 41 (01) 36-43.
  • 7 Fieschi M, Dufour JC, Staccini P, Gouvernet J, Bouhaddou O. Medical decision supportsystems: old dilemmas and new paradigms?. Methods Inf Med 2003; 42 (03) 190-8.
  • 8 Leonard KJ. Total quality in information systems management: issues for the health care industry. Methods Inf Med 1998; 37 (02) 156-60.
  • 9 Nykänen P, Karimaa E. Success and failure factors in the regional health information system design process – results from a constructive evaluation study. Methods Inf Med 2006; 45 (01) 85-9.
  • 10 Stefanelli M. Knowledge and process management in health care organizations. Methods Inf Med 2004; 43 (05) 525-35.
  • 11 Garde S, Wolff AC, Kutscha U, Wetter T, Knaup P. CSI-ISC – Concepts for smooth integration of health care information system components into established processes of patient care. Methods Inf Med 2006; 45 (01) 10-8.
  • 12 OECD. A disease-based comparison of health systems. What is best and at what cost?. Paris: Organisation for Economic Co-operation and Development; 2003
  • 13 Grady ML, Schwartz HA. Medical effectiveness research data methods.. Rockville: Agency for Health Care Policy and Research; 1992
  • 14 Kumar A, Smith B, Pisanelli DM, Gangemi A, Stefanelli M. Clinical guidelines as plans – an ontological theory. Methods Inf Med 2006; 45 (02) 204-10.
  • 15 Ghali WA, Hall RE, Ash AS, Rosen AK, Moskowitz MA. Evaluation of complication rates after coronary artery bypass surgery using administrative data. Methods Inf Med 1998; 37 (02) 192-200.
  • 16 D’Hoore W, Sicotte C, Tilquin C. Risk adjustment in outcome assessment: the Charlson comorbidity index. Methods Inf Med 1993; 32 (05) 382-7.
  • 17 O’Connell RL, Lim LL. Utility of the Charlson comorbidity index computed from routinely collected hospital discharge diagnosis codes. Methods Inf Med 2000; 39 (01) 7-11.
  • 18 Iezzoni LI. Risk Adjustment for measuring health care outcomes.. Third edition. Chigago: Health Administration Press; 2003
  • 19 Treml J, Kroker PB. Hip fracture audit: time for standards to be evidence-based. Int Orthop 2000; 24 (04) 181-3.
  • 20 Parker MJ, Currie CT, Mountain JA, Thorngren KG. Standardised audit of hip fracture in Europe (SAHFE). Hip Int 1998; 8 (01) 10-5.
  • 21 Heikkinen T, Willig R, Hänninen A, Koskinen K, Mannismäki P, Alavaikko A. et al. Hip fractures in Finland – a comparison of patient characteristics and outcomes in six hospitals. Scand J Surg 2004; 93 (03) 234-40.
  • 22 Nurmi I, Narinen A, Lüthje P, Tanninen S. Cost analysis of hip fracture treatment among the elderly for the public health services: a 1-year prospective study in 106 consecutive patients. Arch Orthop Trauma Surg 2003; 123 (10) 551-4.
  • 23 Nurmi I, Narinen A, Lüthje P, Tanninen S. Functional outcome and survival after hip fracture in elderly: apropective study of 106 consecutive patients. J Orthopaed Traumatol 2004; 5 (01) 7-14.
  • 24 Powell AE, Davies HT, Thomson RG. Using routine comparative data to assess the quality of health care: understanding and avoiding common pitfalls. Qual Saf Health Care 2003; 12 (02) 122-8.
  • 25 Rissanen P, Sund R, Nordback I, Rousi T, Idänpään-Heikkilä U. Register based effectiveness measurement in the case of hip fracture treatment (in Finnish).. Helsinki: National Research and Development Centre for Welfare and Health; 2002
  • 26 Sund R. Utilisation of administrative registers using scientific knowledge discovery. Intelligent Data Analysis 2003; 7 (06) 501-19.
  • 27 Sund R, Liski A. Quality effects of operative delay on mortality in hip fracture treatment. Qual Saf Health Care 2005; 14 (05) 371-7.
  • 28 Gissler M, Haukka J. Finnish health and social welfare registers in epidemiological research. Norsk Epidemiologi 2004; 14 (01) 113-20.
  • 29 Roos LL, Wajda A. Record linkage strategies. Part I: Estimating information and evaluating approaches. Methods Inf Med 1991; 30 (02) 117-23.
  • 30 Wajda A, Roos LL, Layefsky M, Singleton JA. Record linkage strategies: Part II. Portable software and deterministic matching. Methods Inf Med 1991; 30 (03) 210-4.
  • 31 Oberaigner W, Stuhlinger W. Record linkage in the Cancer Registry of Tyrol, Austria. Methods Inf Med. 2005; 44 (05) 626-30.
  • 32 Keskimäki I, Aro S. Accuracy of data on diagnoses, procedures and accidents in the Finnish Hospital Discharge Register. Int J Health Sci 1991; 2 (01) 15-21.
  • 33 Lahti RA, Penttilä A. The validity of death certificates: routine validation of death certification and its effects on mortality statistics. Forensic Sci Int 2001; 115 1-2 15-32.
  • 34 Leppälä JM, Virtamo J, Heinonen OP. Validation of stroke diagnosis in the National Hospital Discharge Register and the Register of Causes of Death in Finland. Eur J Epidemiol 1999; 15 (02) 155-60.
  • 35 Pajunen P, Koukkunen H, Ketonen M, Jerkkola T, Immonen-Räihä P, Kärjä-Koskenkari P. et al. The validity of the Finnish Hospital Discharge Register and Causes of Death Register data on coronary heart disease. Eur J Cardiovasc Prev Rehabil 2005; 12 (02) 132-7.
  • 36 Lüthje P, Nurmi I, Kataja M, Heliövaara M, Santavirta S. Incidence of pelvic fractures in Finland in 1988. Acta Orthop Scand 1995; 66 (03) 245-8.
  • 37 Huusko TM, Karppi P, Avikainen V, Kautiainen H, Sulkava R. The changing picture of hip fractures: dramatic change in age distribution and no change in age-adjusted incidence within 10 years in Central Finland. Bone 1999; 24 (03) 257-9.
  • 38 Sorensen HT, Sabroe S, Olsen J. A framework for evaluation of secondary data sources for epidemiological research. Int J Epidemiol 1996; 25 (02) 435-42.
  • 39 Winthereik BR. “We fill in our working understanding”: on codes, classifications and the production of accurate data. Methods Inf Med 2003; 42 (04) 489-96.
  • 40 Hornbrook MC, Hurtado AV, Johnson RE. Health care episodes: definition, measurement and use. Med Care Rev 1985; 42 (02) 163-218.
  • 41 Chin MH, Muramatsu N. What is the quality of quality of medical care measures? Rashomon-like relativism and real-world applications. Perspect Biol Med 2003; 46 (01) 5-20 discussion 21-3.
  • 42 Audige L, Bhandari M, Kellam J. Howreliable are reliability studies of fracture classifications? A systematic review of their methodologies. Acta Orthop Scand 2004; 75 (02) 184-94.
  • 43 Zhang J. Representations of health concepts: a cognitive perspective. J Biomed Inform 2002; 35 (01) 17-24.
  • 44 Wray NP, Ashton CM, Kuykendall DH, Hollingsworth JC. Using administrative databases to evaluate the quality of medical care: a conceptual framework. Soc Sci Med 1995; 40 (12) 1707-15.
  • 45 Kane RL. Understanding health care outcomes research.. Gaithersburg: Aspen Publishers; 1997
  • 46 Shroyer AL, London MJ, Sethi GK, Marshall G, Grover FL, Hammermeister KE. Relationships between patient-related risk factors, processes, structures, and outcomes of cardiac surgical care. Conceptual models. Med Care 1995; 33 (Suppl. 10) OS26-34.
  • 47 Hripcsak G, Heitjan DF. Measuring agreement in medical informatics reliability studies. J Biomed Inform 2002; 35 (02) 99-110.
  • 48 Andres AM, Marzo PF. Delta: A new measure of agreement between two raters. Br J Math Stat Psychol 2004; 57 (01) 1-19.
  • 49 Andres AM, Marzo PF. Chance-corrected measures of reliability and validity in Κ χ K tables. Stat Methods Med Res 2005; 14 (05) 473-92.
  • 50 Audige L, Bhandari M, Hanson B, Kellam J. A concept for the validation of fracture classifications. J Orthop Trauma 2005; 19 (06) 401-6.
  • 51 Björkgren MA, Fries BE, Häkkinen U, Brommels M. Case-mix adjustment and efficiencymeasurement. Scand J Public Health 2004; 32 (06) 464-71.
  • 52 Schnabel M. Representing and processing medical knowledge using formal concept analysis. Methods Inf Med 2002; 41 (02) 160-7.
  • 53 Degoulet P, Fieschi M, Chatellier G. Decision support systems from the standpoint of knowledge representation. Methods Inf Med 1995; 34 1-2 202-8.
  • 54 Brandt CA, Sun K, Charpentier P, Nadkarni PM. Integration of Web-based and PC-based clinical research databases. Methods Inf Med 2004; 43 (03) 287-95.
  • 55 Brandt CA, Cohen DB, Shifinan MA, Miller PL, Nadkarni PM, Frawley SJ. Approaches and informatics tools to assist in the integration of similar clinical research questionnaires. Methods Inf Med 2004; 43 (02) 156-62.
  • 56 van Ginneken AM. Considerations for the representation of meta-data for the support of structured data entry. Methods Inf Med 2003; 42 (03) 226-35.
  • 57 Haux R. Knowledge-based decision support for diagnosis and therapy: on the multiple usability of patient data. Methods Inf Med 1989; 28 (02) 69-77.
  • 58 Berg M, Goorman E. The contextual nature of medical information. Int J Med Inform 1999; 56 1-3 51-60.
  • 59 IWGDMF. International Working Group for Disease Monitoring and Forecasting. Capture-recapture and multiple-record systems estimation I: History and theoretical development. Am J Epidemiol 1995; 142 (10) 1047-58.
  • 60 Brenner H. Application of capture-recapture methods for disease monitoring: potential effects of imperfect record linkage. Methods Inf Med 1994; 33 (05) 502-6.
  • 61 Langley J, Stephenson S, Cryer C, Borman B. Traps for the unwary in estimating person based injury incidence using hospital discharge data. Inj Prev 2002; 8 (04) 332-7.
  • 62 Alsop JC, Langley JD. Determining first admissions in a hospital discharge file via record linkage. Methods Inf Med 1998; 37 (01) 32-7.
  • 63 Los RK, Roukema J, van Ginneken AM, de Wilde M, van der Lei J. Are structured data structured identically? Investigating the uniformity of pediatric patient data recorded using OpenSDE. Methods Inf Med 2005; 44 (05) 631-8.
  • 64 Williams JG. Measuring the completeness and currency of codified clinical information. Methods Inf Med 2003; 42 (04) 482-8.
  • 65 Maly M, Vondra V. Generic versus disease-specific instruments in quality-of-life assessment of chronic obstructive pulmonary disease. Methods Inf Med 2006; 45 (02) 211-5.