Identification of serious adverse events in a large acupuncture trial: Linkage and agreement of primary and secondary data

Background: Large randomized and non-randomized trials in routine care are an important tool to evaluate effectiveness and safety of medical interventions in clinical practice. Complete assessment of serious adverse events (SAEs) is crucial to characterize the safety profile of the drug or intervention under study. We linked data from a large acupuncture trial with hospital claims data to evaluate the agreement of physician-reported SAEs and SAEs identified by using hospital discharge diagnoses. Methods: Patients treated with acupuncture for chronic pain, asthma, allergic rhinitis, or dysmenorrhea within the Acupuncture in Routine Care (ARC) studies were included in this analysis. Patients eligible for ARC had to be insured with a large German statutory health insurance (Techniker Krankenkasse; TK). Acupuncture treatment comprised of a mean number of 10 acupuncture sessions within three months. Physician-reported SAEs were assessed by a follow-up questionnaire. We used hospital claims data to identify SAEs that led to hospitalisation. Cohen's Kappa was calculated as a measure of inter-observer agreement between these two data sources. Results: A total of 42.272 patients (67.5% women, 32.5% men) treated by 6557 different physicians were included in the analysis. Using hospital claims data, 566 patients with at least one SAE during follow-up were identified. For 22 of them (3.9%), the physician also reported a SAE. For 78 patients, only physician-based reporting, but not hospital claims indicated occurrence of a SAE. The kappa for overall inter-observer agreement was 0.06 (95% confidence interval 0.04 to 0.09). Conclusion: In this acupuncture trial, agreement between physician-reported SAEs and SAEs identified via hospital discharge diagnoses was very low, most probably due to substantial SAE underreporting by physicians. Especially in large trials in routine care, linkage with secondary data sources may improve completeness and validity of data.