Comparative Effectiveness Research Using Randomized Trials and Observational Studies: Validity and Feasibility Considerations

Abstract

In comparative effectiveness research (CER), ensuring internal, construct, and external validity is crucial. Internal validity determines whether observed outcomes are causally linked to an intervention; construct validity assesses whether a study measures what it intends to; and external validity relates to generalizability in routine practice. Double-blind randomized trials optimize internal validity by minimizing bias and confounding, while construct validity is strengthened through pre-specified protocols and standardized data collection. However, controlled conditions limit external validity. Pragmatic RCTs improve generalizability but may compromise internal validity due to open-label designs. Observational CER studies—including observational studies following the target trial emulation framework—offer broader external validity and feasibility in less time and at lower cost. However, due to lack of random assignment, these studies are susceptible to measured and unmeasured confounding. Several techniques help mitigate these concerns, including a detailed pre-specified protocol, tools such as propensity score matching to balance measured confounders, falsification endpoint testing for assessing the presence of unmeasured confounders, and quasi-experimental designs (including instrumental variable analysis), which may be able to address both. Pre-specified sensitivity analyses and triangulation with complementary data sources further enhance robustness. Construct validity in observational CER depends on accurate patient profiling and validated computational phenotypes for identifying patients, exposures, and outcomes. Thoughtful study design and analytic rigor are essential for balancing these validity considerations. This brief review highlights these issues with examples from thrombosis research.

Keywords

randomized controlled trials - comparative effectiveness - target trial framework - thrombosis - validity

Publication History

Received: 12 May 2025

Accepted: 23 July 2025

Accepted Manuscript online:
24 July 2025

Article published online:
08 August 2025

• References

• 1 Ford I, Norrie J. Pragmatic trials. N Engl J Med 2016; 375 (05) 454-463
• 2 Kang DS, Yang PS, Kim D. et al. Racial differences in ischemic and hemorrhagic stroke: an ecological epidemiological study. Thromb Haemost 2024; 124 (09) 883-892
• 3 Guo Y, Lane DA, Wang L. et al; mAF-App II Trial Investigators. Mobile health technology to improve care for patients with atrial fibrillation. J Am Coll Cardiol 2020; 75 (13) 1523-1534
• 4 Chu M, Zhang S, Gong J. et al; MIRACLE-AF Investigators. Telemedicine-based integrated management of atrial fibrillation in village clinics: a cluster randomized trial. Nat Med 2025; 31 (04) 1276-1285
• 5 Bikdeli B, Khairani CD, Barns BM. et al. Women's representation in venous thromboembolism randomized trials and registries: the illustrative example of direct oral anticoagulants for acute treatment. Contemp Clin Trials 2022; 115: 106714
• 6 You SC, Krumholz HM. The evolution of evidence-based medicine: when the magic of the randomized clinical trial meets real-world data. Circulation 2022; 145 (02) 107-109
• 7 Welsh J, Lu Y, Dhruva SS. et al. Age of data at the time of publication of contemporary clinical trials. JAMA Netw Open 2018; 1 (04) e181065
• 8 Mauri L, Kereiakes DJ, Yeh RW. et al; DAPT Study Investigators. Twelve or 30 months of dual antiplatelet therapy after drug-eluting stents. N Engl J Med 2014; 371 (23) 2155-2166
• 9 Butala NM, Faridi KF, Tamez H. et al. Estimation of DAPT study treatment effects in contemporary clinical practice: findings from the EXTEND-DAPT study. Circulation 2022; 145 (02) 97-106
• 10 Erdogan E, Bajaj R, Lansky A, Mathur A, Baumbach A, Bourantas CV. Intravascular imaging for guiding in-stent restenosis and stent thrombosis therapy. J Am Heart Assoc 2022; 11 (22) e026492
• 11 Bikdeli B, Ujueta F, Rashedi S. et al. Antiplatelet and anticoagulant therapy in the 2025 ACC/AHA guideline for acute coronary syndromes: key recommendations. JACC 2025 85. 22 . In-Press
• 12 Young K. FDA Loosens Restrictions on Metformin Use in Patients with Mild-to-Moderate Kidney Dysfunction. 2016 . Accessed April 29, 2025 at: https://www.jwatch.org/fw111424/2016/04/11/fda-loosens-restrictions-metformin-use-patients-with-mild
• 13 Kim JY, Kim SH, Myong JP. et al. Outcomes of direct oral anticoagulants in patients with mitral stenosis. J Am Coll Cardiol 2019; 73 (10) 1123-1131
• 14 Connolly SJ, Karthikeyan G, Ntsekhe M. et al; INVICTUS Investigators. Rivaroxaban in rheumatic heart disease-associated atrial fibrillation. N Engl J Med 2022; 387 (11) 978-988
• 15 Bejjani A, Bikdeli B. Direct oral anticoagulants: quick primer on when to use and when to avoid. Thromb Haemost 2025; 125 (07) 611-617
• 16 Hernán MA, Robins JM. Using big data to emulate a target trial when a randomized trial is not available. Am J Epidemiol 2016; 183 (08) 758-764
• 17 Hansford HJ, Cashin AG, Jones MD. et al. Reporting of observational studies explicitly aiming to emulate randomized trials: a systematic review. JAMA Netw Open 2023; 6 (09) e2336023
• 18 Dahabreh IJ, Bibbins-Domingo K. Causal inference about the effects of interventions from observational studies in medical journals. JAMA 2024; 331 (21) 1845-1853
• 19 Suchard MA, Schuemie MJ, Krumholz HM. et al. Comprehensive comparative effectiveness and safety of first-line antihypertensive drug classes: a systematic, multinational, large-scale analysis. Lancet 2019; 394 (10211): 1816-1826
• 20 Wallach JD, Deng Y, Polley EC. et al. Assessing the use of observational methods and real-world data to emulate ongoing randomized controlled trials. Clin Trials 2023; 20 (06) 689-698
• 21 Wang SV, Schneeweiss S, Franklin JM. et al; RCT-DUPLICATE Initiative. Emulation of randomized clinical trials with nonrandomized database analyses: results of 32 clinical trials. JAMA 2023; 329 (16) 1376-1385
• 22 You SC, Rho Y, Bikdeli B. et al. Association of ticagrelor vs clopidogrel with net adverse clinical events in patients with acute coronary syndrome undergoing percutaneous coronary intervention. JAMA 2020; 324 (16) 1640-1650
• 23 Hanberg JS, Fu X, Wang X. et al. Effectiveness of a fourth dose of COVID-19 mRNA vaccine in patients with systemic autoimmune rheumatic diseases using disease-modifying antirheumatic drugs: an emulated target trial. Lancet Rheumatol 2024; 6 (01) e21-e30
• 24 Xie Y, Bowe B, Xian H, Loux T, McGill JB, Al-Aly Z. Comparative effectiveness of SGLT2 inhibitors, GLP-1 receptor agonists, DPP-4 inhibitors, and sulfonylureas on risk of major adverse cardiovascular events: emulation of a randomised target trial using electronic health records. Lancet Diabetes Endocrinol 2023; 11 (09) 644-656
• 25 Kutcher SA, Brophy JM, Banack HR, Kaufman JS, Samuel M. Emulating a randomised controlled trial with observational data: an introduction to the target trial framework. Can J Cardiol 2021; 37 (09) 1365-1377
• 26 Bikdeli B, Lo YC, Khairani CD. et al. Developing validated tools to identify pulmonary embolism in electronic databases: rationale and design of the PE-EHR+ study. Thromb Haemost 2023; 123 (06) 649-662
• 27 Bikdeli B, Khairani CD, Bejjani A. et al; PE-EHR+ Investigators. Validating International Classification of Diseases Code 10th Revision algorithms for accurate identification of pulmonary embolism. J Thromb Haemost 2025; 23 (02) 556-564
• 28 Wallach JD, Zhang AD, Skydel JJ. et al. Feasibility of using real-world data to emulate postapproval confirmatory clinical trials of therapeutic agents granted US Food and Drug Administration accelerated approval. JAMA Netw Open 2021; 4 (11) e2133667
• 29 Cunningham JW, Singh P, Reeder C. et al. Natural language processing for adjudication of heart failure in a multicenter clinical trial: a secondary analysis of a randomized clinical trial. JAMA Cardiol 2024; 9 (02) 174-181
• 30 Hubbard RA, Gatsonis CA, Hogan JW, Hunter DJ, Normand ST, Troxel AB. “Target trial emulation” for observational studies—potential and pitfalls. N Engl J Med 2024; 391 (21) 1975-1977
• 31 Hernán MA, Wang W, Leaf DE. Target trial emulation: a framework for causal inference from observational data. JAMA 2022; 328 (24) 2446-2447
• 32 Monteleone P, Ahern R, Banerjee S. et al. Modern treatment of pulmonary embolism (USCDT vs MT): results from a real-world, big data analysis (REAL-PE). J Soc Cardiovasc Angiogr Interv 2023; 3 (01) 101192
• 33 Cohen AT, Sah J, Dhamane AD. et al. Effectiveness and safety of apixaban versus warfarin in venous thromboembolism patients with chronic kidney disease. Thromb Haemost 2022; 122 (06) 926-938
• 34 Chowdhury KR, Michaud J, Yu OHY, Yin H, Azoulay L, Renoux C. Effectiveness and safety of apixaban versus rivaroxaban in patients with atrial fibrillation and type 2 diabetes mellitus. Thromb Haemost 2022; 122 (10) 1794-1803
• 35 Weycker D, Li X, Wygant GD. et al. Effectiveness and safety of apixaban versus warfarin as outpatient treatment of venous thromboembolism in U.S. clinical practice. Thromb Haemost 2018; 118 (11) 1951-1961
• 36 Butala NM, Makkar R, Secemsky EA. et al. Cerebral embolic protection and outcomes of transcatheter aortic valve replacement: results from the transcatheter valve therapy registry. Circulation 2021; 143 (23) 2229-2240
• 37 Kapadia SR, Kodali S, Makkar R. et al; SENTINEL Trial Investigators. Protection against cerebral embolism during transcatheter aortic valve replacement. J Am Coll Cardiol 2017; 69 (04) 367-377
• 38 Granger CB, Alexander JH, McMurray JJ. et al; ARISTOTLE Committees and Investigators. Apixaban versus warfarin in patients with atrial fibrillation. N Engl J Med 2011; 365 (11) 981-992
• 39 Volgman AS, Benjamin EJ, Curtis AB. et al; American College of Cardiology Committee on Cardiovascular Disease in Women. Women and atrial fibrillation. J Cardiovasc Electrophysiol 2021; 32 (10) 2793-2807
• 40 Wallentin L, Becker RC, Budaj A. et al; PLATO Investigators. Ticagrelor versus clopidogrel in patients with acute coronary syndromes. N Engl J Med 2009; 361 (11) 1045-1057
• 41 Jones WS, Mulder H, Wruck LM. et al; ADAPTABLE Team. Comparative effectiveness of aspirin dosing in cardiovascular disease. N Engl J Med 2021; 384 (21) 1981-1990
• 42 Erlinge D, Omerovic E, Fröbert O. et al. Bivalirudin versus heparin monotherapy in myocardial infarction. N Engl J Med 2017; 377 (12) 1132-1142