Inter-Observer Variation in Decision-Making Regarding Patients on Chronic Hemodialysis: A Study Using the Kappa Index

Abstract

To examine inter-observer variation in the monitoring of patients with chronic disease, four nephrologists independently assessed 62 patients on maintenance hemodialysis. Deviation from normal was determined for adequacy of dialysis, protein intake, and metabolic state. The kappa-index, which allows to adjust for chance agreement, was used to analyze each monitoring diagnosis. Low agreement was found on decisions concerning adequacy of dialysis (kappa 0.12-0.26), while agreement was higher about protein intake (kappa 0.21-0.46), and metabolic state (kappa 0.24-0.52). Two physicians classified no patient as overdialyzed, while 16-18% were thus categorized by the other two. Routines for review of recent medical history also differed significantly between the physicians. Measures are needed to increase the reliability of decisions regarding the monitoring of chronic hemodialysis. A long lasting physician-patient relationship is not a sufficient prerequisite for diminishing decision variation. Medical audit as part of the clinical routine, and use of additional sources of information, exemplified by urea kinetic modeling, are discussed.

Key-Words

Medical Decision Making - Inter-Observer Variation - Hemodialysis - Kappa Analysis - Decision Support

^* The kappa-index values can be interpreted as uncertain agreement for values in range (0.00–0.19), low agreement (0.20–0.39), moderate agreement (0040–0.59), substantial agreement (0.60–0.79) and almost perfect agreement (> 0.80) [5].

• REFERENCES

• 1 Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas 1960; 20: 37-46.
• 2 Koran LM. The reliability of clinical methods, data and judgement. N Engl J Med. 1975 293. 642-6, 695-700..
• 3 Corazza GR, Bonvicini F, Frazzoni M, Gatto M, Gasbarrini G. Observer variation in assessment of jejunal biopsy specimen. A comparison between subjective criteria and morphometric measurement. Gastroenterology 1982; 83: 1217-22.
• 4 Timpka T, Bjurulf P. The semantics of diagnosis and management of genitourinary infections: A cross-specialty study. Fam Pract 1989; 06: 279-85.
• 5 LandiS JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33: 159-74.
• 6 Hartling OJ, Höilund-Carisen PG, Hesse B. et al. Evaluation of intravenous radionuclide angiography in detection of occlusive disease of the iliac arteries. Europ J Radiol 1984; 04: 205-9.
• 7 Gjörup T, Agner E, Jensen LB, Jensen AM, MöIlman KM. The endoscopic diagnosis of duodenal ulcer disease. Scand J Gastroenterology 1986; 21: 261-7.
• 8 Theodossi A, Knill-Jones RP, Skene A. et al. Inter-observer variation of symptoms and signs in jaundice. Liver 1981; 01: 21-32.
• 9 Quinton WE, Dillard D, Scribner BH. Cannulation of blood vessels for prolonged hemodialysis. Trans Am Soc Artif Intern Organs 1963; 09: 67-72.
• 10 Schupak E, Merrill JP. Experience with long-term intermittent hemodialysis. Ann Intern Med 1965; 62: 509-14.
• 11 Theodossi A, Skene AM, Portman B. et al. Observer variation in assessment of liver biopsies including analyses by kappa statistics. Gastroenterology 1980; 79: 232-41.
• 12 Avram MM, Slater PA, Gan A. et al. PredialysisBUN and creatinine do not predict adequate dialysis, clinical rehabilitation, or longevity. Kidney Int 1985; 28 (Suppl. 17) S100-5.
• 13 Shapiro JI, Argy wP, Rakowski TA, Chester A, Siemsen AS. Schreiner GE. The unsuitability of BUN as a criterion for prescription of dialysis. Trans Am Soc Artif Intern Organs 1983; 29: 286-92.
• 14 Gotch FA, Sargent JA. A mechanistic analysis of the National Cooperative Dialysis Study (NCDS). Kidney Int 1985; 28: 526-34.
• 15 Dougirdas JT, Dumler F, Zasuwa GA, Levin Nw. Chronic hemodialysis prescription. In: Daugirdas IT, Ing TS. eds. Handbook of Dialysis. Boston: Little, Brown and Co; 1988: 72-86.
• 16 Musch DC, Higgins ITT, Landis JR. Some factors influencing interobserver variation in classifying simple pneumoconiosis. Br J Ind Med 1985; 42: 346-9.
• 17 Shaw CD, Costain DW. Guidelines for medical audit: seven principles. Br Med J 1989; 299: 488-9.
• 18 Gjörup T, Hamberg O, Knudsen J. et al. Does the patient appear acutely or chronically ill? An interobserver study of global assessment of hospital patients. Acta Med Scand 1982; 212: 325-8.
• 19 Gjorup T. The kappa coefficient and the prevalence of a diagnosis. Meth Inform Med 1988; 27: 184-6.
• 20 Buur T. An advanced, user-friendly microcomputer program for hemodialysis kinetics. Adv Exp Med Bioi 1987; 223: 239-44.