PDF herunterladen
Open Access
CC BY 4.0 · Yearb Med Inform 2024; 33(01): 127-142
DOI: 10.1055/s-0044-1800732
Section 4: Clinical Research Informatics
Survey

Clinical Research Informatics: a Decade-in-Review

Authors

  • Christel Daniel

    1   AP-HP, France
    2   Sorbonne Université, INSERM UMR_S 1142, LIMICS, F-75006, Paris, France

  • Peter J. Embí

    3   Vanderbilt University Medical Center, Department of Biomedical Informatics, Nashville, Tennessee, USA
Weitere Informationen
Auch verfügbar auf
  Lizenzen und Reprints
Preview

Summary

Background: Clinical Research Informatics (CRI) is a subspeciality of biomedical informatics that has substantially matured during the last decade. Advances in CRI have transformed the way clinical research is conducted. In recent years, there has been growing interest in CRI, as reflected by a vast and expanding scientific literature focused on the topic. The main objectives of this review are: 1) to provide an overview of the evolving definition and scope of this biomedical informatics subspecialty over the past 10 years; 2) to highlight major contributions to the field during the past decade; and 3) to provide insights about more recent CRI research trends and perspectives.

Methods: We adopted a modified thematic review approach focused on understanding the evolution and current status of the CRI field based on literature sources identified through two complementary review processes (AMIA CRI year-in-review/IMIA Yearbook of Medical Informatics) conducted annually during the last decade.

Results: More than 1,500 potentially relevant publications were considered, and 205 sources were included in the final review. The review identified key publications defining the scope of CRI and/or capturing its evolution over time as illustrated by impactful tools and methods in different categories of CRI focus. The review also revealed current topics of interest in CRI and prevailing research trends.

Conclusion: This scoping review provides an overview of a decade of research in CRI, highlighting major changes in the core CRI discoveries as well as increasingly impactful methods and tools that have bridged the principles-to-practice gap. Practical CRI solutions as well as examples of CRI-enabled large-scale, multi-organizational and/or multi-national research projects demonstrate the maturity of the field. Despite the progress demonstrated, some topics remain challenging, highlighting the need for ongoing CRI development and research, including the need of more rigorous evaluations of CRI solutions and further formalization and maturation of CRI services and capabilities across the research enterprise.

Keywords

Clinical Research Informatics - Biomedical Research - Clinical Trials - Informatics - Literature Review


Publikationsverlauf

Artikel online veröffentlicht:
08. April 2025

© 2024. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution License, permitting unrestricted use, distribution, and reproduction so long as the original work is properly cited. (https://creativecommons.org/licenses/by/4.0/)

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 Embi PJ, Payne PRO. Clinical research informatics: challenges, opportunities and definition for an emerging domain. J Am Med Inform Assoc JAMIA 2009;16:316–27. https://doi.org/10.1197/jamia.M3005.
    PubMed
  • 2 Weng C, Kahn MG. Clinical Research Informatics for Big Data and Precision Medicine. Yearb Med Inform 2016:211–8. https://doi.org/10.15265/IY-2016-019.
    PubMed
  • 3 Johnson SB. Clinical Research Informatics: Supporting the Research Study Lifecycle. Yearb Med Inform 2017;26:193–200. https://doi.org/10.15265/IY-2017-022.
    PubMed
  • 4 Embi PJ. Clinical research informatics: survey of recent advances and trends in a maturing field. Yearb Med Inform 2013;8:178–84.
    PubMed
  • 5 Solomonides A. Review of Clinical Research Informatics. Yearb Med Inform 2020;29:193–202. https://doi.rg/10.1055/s-0040-1701988.
    PubMed
  • 6 Murphy SN, Dubey A, Embi PJ, Harris PA, Richter BG, Turisco F, et al. Current state of information technologies for the clinical research enterprise across academic medical centers. Clin Transl Sci 2012;5:281–4. https://doi.org/10.1111/j.1752-8062.2011.00387.x.
    PubMed
  • 7 Tasneem A, Aberle L, Ananth H, Chakraborty S, Chiswell K, McCourt BJ, et al. The database for aggregate analysis of ClinicalTrials.gov (AACT) and subsequent regrouping by clinical specialty. PloS One 2012;7:e33677. https://doi.org/10.1371/journal.pone.0033677.
    PubMed
  • 8 Bansal A, Chamberlain R, Karr S, Kwasa S, McLaughlin B, Nguyen B, et al. A 21 CFR Part 11 compliant graphically based electronic system for clinical research documentation. J Med Syst 2012;36:1661–72. https://doi.org/10.1007/s10916-010-9627-4.
    PubMed
  • 9 Marsolo K. Informatics and operations-let's get integrated. J Am Med Inform Assoc JAMIA 2013;20:122–4. https://doi.org/10.1136/amiajnl-2012-001194.
    PubMed
  • 10 Coorevits P, Sundgren M, Klein GO, Bahr A, Claerhout B, Daniel C, et al. Electronic health records: new opportunities for clinical research. J Intern Med 2013;274:547–60. https://doi.org/10.1111/joim.12119.
    PubMed
  • 11 Köpcke F, Kraus S, Scholler A, Nau C, Schüttler J, Prokosch H-U, et al. Secondary use of routinely collected patient data in a clinical trial: an evaluation of the effects on patient recruitment and data acquisition. Int J Med Inf 2013;82:185–92. https://doi.org/10.1016/j.ijmedinf.2012.11.008.
    PubMed
  • 12 Weng C, Li Y, Berhe S, Boland MR, Gao J, Hruby GW, et al. An Integrated Model for Patient Care and Clinical Trials (IMPACT) to support clinical research visit scheduling workflow for future learning health systems. J Biomed Inform 2013;46:642–52. https://doi.org/10.1016/j.jbi.2013.05.001.
    PubMed
  • 13 Shivade C, Raghavan P, Fosler-Lussier E, Embi PJ, Elhadad N, Johnson SB, et al. A review of approaches to identifying patient phenotype cohorts using electronic health records. J Am Med Inform Assoc JAMIA 2014;21:221–30. https://doi.org/10.1136/amiajnl-2013-001935.
    PubMed
  • 14 Fernández-Breis JT, Maldonado JA, Marcos M, Legaz-García M del C, Moner D, Torres-Sospedra J, et al. Leveraging electronic healthcare record standards and semantic web technologies for the identification of patient cohorts. J Am Med Inform Assoc JAMIA 2013;20:e288-296. https://doi.org/10.1136/amiajnl-2013-001923.
    PubMed
  • 15 Sim I, Tu SW, Carini S, Lehmann HP, Pollock BH, Peleg M, et al. The Ontology of Clinical Research (OCRe): an informatics foundation for the science of clinical research. J Biomed Inform 2014;52:78–91. https://doi.org/10.1016/j.jbi.2013.11.002.
    PubMed
  • 16 Doods J, Botteri F, Dugas M, Fritz F, EHR4CR WP7. A European inventory of common electronic health record data elements for clinical trial feasibility. Trials 2014;15:18. https://doi.org/10.1186/1745-6215-15-18.
    PubMed
  • 17 Schreiweis B, Trinczek B, Köpcke F, Leusch T, Majeed RW, Wenk J, et al. Comparison of electronic health record system functionalities to support the patient recruitment process in clinical trials. Int J Med Inf 2014;83:860–8. https://doi.org/10.1016/j.ijmedinf.2014.08.005.
    PubMed
  • 18 Richesson RL, Horvath MM, Rusincovitch SA. Clinical research informatics and electronic health record data. Yearb Med Inform 2014;9:215–23. https://doi.org/10.15265/IY-2014-0009.
    PubMed
  • 19 Miotto R, Weng C. Case-based reasoning using electronic health records efficiently identifies eligible patients for clinical trials. J Am Med Inform Assoc JAMIA 2015;22:e141-150. https://doi.org/10.1093/jamia/ocu050.
    PubMed
  • 20 Soto-Rey I, Trinczek B, Girardeau Y, Zapletal E, Ammour N, Doods J, et al. Efficiency and effectiveness evaluation of an automated multi-country patient count cohort system. BMC Med Res Methodol 2015;15:44. https://doi.org/10.1186/s12874-015-0035-9.
    PubMed
  • 21 Wu DTY, Hanauer DA, Mei Q, Clark PM, An LC, Proulx J, et al. Assessing the readability of ClinicalTrials.gov. J Am Med Inform Assoc JAMIA 2016;23:269–75. https://doi.org/10.1093/jamia/ocv062.
    PubMed
  • 22 Dugas M. Clinical Research Informatics: Recent Advances and Future Directions. Yearb Med Inform 2015;10:174–7. https://doi.org/10.15265/IY-2015-010.
    PubMed
  • 23 De Moor G, Sundgren M, Kalra D, Schmidt A, Dugas M, Claerhout B, et al. Using electronic health records for clinical research: the case of the EHR4CR project. J Biomed Inform 2015;53:162–73. https://doi.org/10.1016/j.jbi.2014.10.006.
    PubMed
  • 24 Beresniak A, Schmidt A, Proeve J, Bolanos E, Patel N, Ammour N, et al. Cost-benefit assessment of using electronic health records data for clinical research versus current practices: Contribution of the Electronic Health Records for Clinical Research (EHR4CR) European Project. Contemp Clin Trials 2016;46:85–91. https://doi.org/10.1016/j.cct.2015.11.011.
    PubMed
  • 25 Ateya MB, Delaney BC, Speedie SM. The value of structured data elements from electronic health records for identifying subjects for primary care clinical trials. BMC Med Inform Decis Mak 2016;16:1. https://doi.org/10.1186/s12911-016-0239-x.
    PubMed
  • 26 Eubank MH, Hyman DM, Kanakamedala AD, Gardos SM, Wills JM, Stetson PD. Automated eligibility screening and monitoring for genotype-driven precision oncology trials. J Am Med Inform Assoc JAMIA 2016;23:777–81. https://doi.org/10.1093/jamia/ocw020.
    PubMed
  • 27 Mastellos N, Bliźniuk G, Czopnik D, McGilchrist M, Misiaszek A, Bródka P, et al. Feasibility and acceptability of TRANSFoRm to improve clinical trial recruitment in primary care. Fam Pract 2016;33:186–91. https://doi.org/10.1093/fampra/cmv102.
    PubMed
  • 28 Hruby GW, Hoxha J, Ravichandran PC, Mendonça EA, Hanauer DA, Weng C. A data-driven concept schema for defining clinical research data needs. Int J Med Inf 2016;91:1–9. https://doi.org/10.1016/j.ijmedinf.2016.03.008.
    PubMed
  • 29 Girardeau Y, Doods J, Zapletal E, Chatellier G, Daniel C, Burgun A, et al. Leveraging the EHR4CR platform to support patient inclusion in academic studies: challenges and lessons learned. BMC Med Res Methodol 2017;17:36. https://doi.org/10.1186/s12874-017-0299-3.
    PubMed
  • 30 Krischer J, Cronholm PF, Burroughs C, McAlear CA, Borchin R, Easley E, et al. Experience With Direct-to-Patient Recruitment for Enrollment Into a Clinical Trial in a Rare Disease: A Web-Based Study. J Med Internet Res 2017;19:e50. https://doi.org/10.2196/jmir.6798.
    PubMed
  • 31 Obeid JS, Beskow LM, Rape M, Gouripeddi R, Black RA, Cimino JJ, et al. A survey of practices for the use of electronic health records to support research recruitment. J Clin Transl Sci 2017;1:246–52. https://doi.org/10.1017/cts.2017.301.
    PubMed
  • 32 Claerhout B, Kalra D, Mueller C, Singh G, Ammour N, Meloni L, et al. Federated electronic health records research technology to support clinical trial protocol optimization: Evidence from EHR4CR and the InSite platform. J Biomed Inform 2019;90:103090. https://doi.org/10.1016/j.jbi.2018.12.004.
    PubMed
  • 33 Meystre SM, Heider PM, Kim Y, Aruch DB, Britten CD. Automatic trial eligibility surveillance based on unstructured clinical data. Int J Med Inf 2019;129:13–9. https://doi.org/10.1016/j.ijmedinf.2019.05.018.
    PubMed
  • 34 Miller HN, Gleason KT, Juraschek SP, Plante TB, Lewis-Land C, Woods B, et al. Electronic medical record-based cohort selection and direct-to-patient, targeted recruitment: early efficacy and lessons learned. J Am Med Inform Assoc JAMIA 2019;26:1209–17. https://doi.org/10.1093/jamia/ocz168.
    PubMed
  • 35 Kim JH, Ta CN, Liu C, Sung C, Butler AM, Stewart LA, et al. Towards clinical data-driven eligibility criteria optimization for interventional COVID-19 clinical trials. J Am Med Inform Assoc JAMIA 2021;28:14–22. https://doi.org/10.1093/jamia/ocaa276.
    PubMed
  • 36 Liu H, Chi Y, Butler A, Sun Y, Weng C. A knowledge base of clinical trial eligibility criteria. J Biomed Inform 2021;117:103771. https://doi.org/10.1016/j.jbi.2021.103771.
    PubMed
  • 37 Rogers JR, Hripcsak G, Cheung YK, Weng C. Clinical comparison between trial participants and potentially eligible patients using electronic health record data: A generalizability assessment method. J Biomed Inform 2021;119:103822. https://doi.org/10.1016/j.jbi.2021.103822.
    PubMed
  • 38 Chen W-C, Li H, Wang C, Lu N, Song C, Tiwari R, et al. Evaluation of diagnostic tests for low prevalence diseases: a statistical approach for leveraging real-world data to accelerate the study. J Biopharm Stat 2021;31:375–90. https://doi.org/10.1080/10543406.2021.1877724.
    PubMed
  • 39 Cheng AC, Duda SN, Taylor R, Delacqua F, Lewis AA, Bosler T, et al. REDCap on FHIR: Clinical Data Interoperability Services. J Biomed Inform 2021;121:103871. https://doi.org/10.1016/j.jbi.2021.103871.
    PubMed
  • 40 Cheng AC, Banasiewicz MK, Johnson JD, Sulieman L, Kennedy N, Delacqua F, et al. Evaluating automated electronic case report form data entry from electronic health records. J Clin Transl Sci 2023;7:e29. https://doi.org/10.1017/cts.2022.514.
    PubMed
  • 41 Eisenstein EL, Zozus MN, Garza MY, Lanham HJ, Adagarla B, Walden A, et al. Assessing clinical site readiness for electronic health record (EHR)-to-electronic data capture (EDC) automated data collection. Contemp Clin Trials 2023;128:107144. https://doi.org/10.1016/j.cct.2023.107144.
    PubMed
  • 42 Mueller C, Herrmann P, Cichos S, Remes B, Junker E, Hastenteufel T, et al. Automated Electronic Health Record to Electronic Data Capture Transfer in Clinical Studies in the German Health Care System: Feasibility Study and Gap Analysis. J Med Internet Res 2023;25:e47958. https://doi.org/10.2196/47958.
    PubMed
  • 43 Meystre SM, Heider PM, Cates A, Bastian G, Pittman T, Gentilin S, et al. Piloting an automated clinical trial eligibility surveillance and provider alert system based on artificial intelligence and standard data models. BMC Med Res Methodol 2023;23:88. https://doi.org/10.1186/s12874-023-01916-6.
    PubMed
  • 44 Lin J, Yu G, Gamalo M. Matching within a hybrid RCT/RWD: framework on associated causal estimands. J Biopharm Stat 2023;33:439–51. https://doi.org/10.1080/10543406.2022.2105346.
    PubMed
  • 45 Lombardo G, Couvert C, Kose M, Begum A, Spiertz C, Worrell C, et al. Electronic health records (EHRs) in clinical research and platform trials: Application of the innovative EHR-based methods developed by EU-PEARL. J Biomed Inform 2023;148:104553. https://doi.org/10.1016/j.jbi.2023.104553.
    PubMed
  • 46 Incerti D, Bretscher MT, Lin R, Harbron C. A meta-analytic framework to adjust for bias in external control studies. Pharm Stat 2023;22:162–80. https://doi.org/10.1002/pst.2266.
    PubMed
  • 47 Carrigan G, Whipple S, Capra WB, Taylor MD, Brown JS, Lu M, et al. Using Electronic Health Records to Derive Control Arms for Early Phase Single-Arm Lung Cancer Trials: Proof-of-Concept in Randomized Controlled Trials. Clin Pharmacol Ther 2020;107:369–77. https://doi.org/10.1002/cpt.1586.
    PubMed
  • 48 Pfiffner PB, Pinyol I, Natter MD, Mandl KD. C3-PRO: Connecting ResearchKit to the Health System Using i2b2 and FHIR. PloS One 2016;11:e0152722. https://doi.org/10.1371/journal.pone.0152722.
    PubMed
  • 49 Sittig DF, Hazlehurst BL, Brown J, Murphy S, Rosenman M, Tarczy-Hornoch P, et al. A survey of informatics platforms that enable distributed comparative effectiveness research using multi-institutional heterogenous clinical data. Med Care 2012;50 Suppl:S49-59. https://doi.org/10.1097/MLR.0b013e318259c02b.
    PubMed
  • 50 Bledsoe MJ, Grizzle WE, Clark BJ, Zeps N. Practical implementation issues and challenges for biobanks in the return of individual research results. Genet Med Off J Am Coll Med Genet 2012;14:478–83. https://doi.org/10.1038/gim.2011.67.
    PubMed
  • 51 O'Meara H, Carr DF, Evely J, Hobbs M, McCann G, van Staa T, et al. Electronic health records for biological sample collection: feasibility study of statin-induced myopathy using the Clinical Practice Research Datalink. Br J Clin Pharmacol 2014;77:831–8. https://doi.org/10.1111/bcp.12269.
    PubMed
  • 52 Forrest CB, Margolis PA, Bailey LC, Marsolo K, Del Beccaro MA, Finkelstein JA, et al. PEDSnet: a National Pediatric Learning Health System. J Am Med Inform Assoc JAMIA 2014;21:602–6. https://doi.org/10.1136/amiajnl-2014-002743.
    PubMed
  • 53 Fleurence RL, Curtis LH, Califf RM, Platt R, Selby JV, Brown JS. Launching PCORnet, a national patient-centered clinical research network. J Am Med Inform AssocJAMIA 2014;21:578–82. https://doi.org/10.1136/amiajnl-2014-002747.
    PubMed
  • 54 van Ommen G-JB, Törnwall O, Bréchot C, Dagher G, Galli J, Hveem K, et al. BBMRI-ERIC as a resource for pharmaceutical and life science industries: the development of biobank-based Expert Centres. Eur J Hum Genet EJHG 2015;23:893–900. https://doi.org/10.1038/ejhg.2014.235.
    PubMed
  • 55 Kass-Hout TA, Xu Z, Mohebbi M, Nelsen H, Baker A, Levine J, et al. OpenFDA: an innovative platform providing access to a wealth of FDA's publicly available data. J Am Med Inform Assoc JAMIA 2016;23:596–600. https://doi.org/10.1093/jamia/ocv153.
    PubMed
  • 56 Johnson AE, Stone DJ, Celi LA, Pollard TJ. The MIMIC Code Repository: enabling reproducibility in critical care research. J Am Med Inform Assoc JAMIA 2018;25:32–9. https://doi.org/10.1093/jamia/ocx084.
    PubMed
  • 57 Denaxas S, Gonzalez-Izquierdo A, Direk K, Fitzpatrick NK, Fatemifar G, Banerjee A, et al. UK phenomics platform for developing and validating electronic health record phenotypes: CALIBER. J Am Med Inform Assoc JAMIA 2019;26:1545–59. https://doi.org/10.1093/jamia/ocz105.
    PubMed
  • 58 Forrest CB, McTigue KM, Hernandez AF, Cohen LW, Cruz H, Haynes K, et al. PCORnet® 2020: current state, accomplishments, and future directions. J Clin Epidemiol 2021;129:60–7. https://doi.org/10.1016/j.jclinepi.2020.09.036.
    PubMed
  • 59 Raisaro JL, Marino F, Troncoso-Pastoriza J, Beau-Lejdstrom R, Bellazzi R, Murphy R, et al. SCOR: A secure international informatics infrastructure to investigate COVID-19. J Am Med Inform Assoc JAMIA 2020;27:1721–6. https://doi.org/10.1093/jamia/ocaa172.
    PubMed
  • 60 Haendel MA, Chute CG, Bennett TD, Eichmann DA, Guinney J, Kibbe WA, et al. The National COVID Cohort Collaborative (N3C): Rationale, design, infrastructure, and deployment. J Am Med Inform Assoc JAMIA 2021;28:427–43. https://doi.org/10.1093/jamia/ocaa196.
    PubMed
  • 61 Kohane IS, Churchill SE, Murphy SN. A translational engine at the national scale: informatics for integrating biology and the bedside. J Am Med Inform Assoc JAMIA 2012;19:181–5. https://doi.org/10.1136/amiajnl-2011-000492.
    PubMed
  • 62 MacKenzie SL, Wyatt MC, Schuff R, Tenenbaum JD, Anderson N. Practices and perspectives on building integrated data repositories: results from a 2010 CTSA survey. J Am Med Inform Assoc JAMIA 2012;19:e119-124. https://doi.org/10.1136/amiajnl-2011-000508.
    PubMed
  • 63 Jensen PB, Jensen LJ, Brunak S. Mining electronic health records: towards better research applications and clinical care. Nat Rev Genet 2012;13:395–405. https://doi.org/10.1038/nrg3208.
    PubMed
  • 64 Hersh WR, Weiner MG, Embi PJ, Logan JR, Payne PRO, Bernstam EV, et al. Caveats for the use of operational electronic health record data in comparative effectiveness research. Med Care 2013;51:S30-37. https://doi.org/10.1097/MLR.0b013e31829b1dbd.
    PubMed
  • 65 Martin Sanchez F, Gray K, Bellazzi R, Lopez-Campos G. Exposome informatics: considerations for the design of future biomedical research information systems. J Am Med Inform Assoc JAMIA 2014;21:386–90. https://doi.org/10.1136/amiajnl-2013-001772.
    PubMed
  • 66 Johnson EK, Broder-Fingert S, Tanpowpong P, Bickel J, Lightdale JR, Nelson CP. Use of the i2b2 research query tool to conduct a matched case-control clinical research study: advantages, disadvantages and methodological considerations. BMC Med Res Methodol 2014;14:16. https://doi.org/10.1186/1471-2288-14-16.
    PubMed
  • 67 Danciu I, Cowan JD, Basford M, Wang X, Saip A, Osgood S, et al. Secondary use of clinical data: the Vanderbilt approach. J Biomed Inform 2014;52:28–35. https://doi.org/10.1016/j.jbi.2014.02.003.
    PubMed
  • 68 Wilhelm EE, Oster E, Shoulson I. Approaches and costs for sharing clinical research data. JAMA 2014;311:1201–2. https://doi.org/10.1001/jama.2014.850.
    PubMed
  • 69 Murphy SN, Herrick C, Wang Y, Wang TD, Sack D, Andriole KP, et al. High throughput tools to access images from clinical archives for research. J Digit Imaging 2015;28:194–204. https://doi.org/10.1007/s10278-014-9733-9.
    PubMed
  • 70 Klann JG, Abend A, Raghavan VA, Mandl KD, Murphy SN. Data interchange using i2b2. J Am Med Inform Assoc JAMIA 2016;23:909–15. https://doi.org/10.1093/jamia/ocv188.
    PubMed
  • 71 Jannot A-S, Zapletal E, Avillach P, Mamzer M-F, Burgun A, Degoulet P. The Georges Pompidou University Hospital Clinical Data Warehouse: A 8-years follow-up experience. Int J Med Inf 2017;102:21–8. https://doi.org/10.1016/j.ijmedinf.2017.02.006.
    PubMed
  • 72 Cuggia M, Combes S. The French Health Data Hub and the German Medical Informatics Initiatives: Two National Projects to Promote Data Sharing in Healthcare. Yearb Med Inform 2019;28:195–202. https://doi.rg/10.1055/s-0039-1677917.
    PubMed
  • 73 Kotecha D, Asstvelbergs FW, Achenbach S, Anker SD, Atar D, Baigent C, et al. CODE-EHR best-practice framework for the use of structured electronic health-care records in clinical research. Lancet Digit Health 2022;4:e757–64. https://doi.org/10.1016/S2589-7500(22)00151-0.
    PubMed
  • 74 Ostropolets A, Albogami Y, Conover M, Banda JM, Baumgartner WA, Blacketer C, et al. Reproducible variability: assessing investigator discordance across 9 research teams attempting to reproduce the same observational study. J Am Med Inform Assoc JAMIA 2023;30:859–68. https://doi.org/10.1093/jamia/ocad009.
    PubMed
  • 75 Knosp BM, Dorr DA, Campion TR. Maturity in enterprise data warehouses for research operations: Analysis of a pilot study. J Clin Transl Sci 2023;7:e70. https://doi.org/10.1017/cts.2023.23.
    PubMed
  • 76 Geissbuhler A, Safran C, Buchan I, Bellazzi R, Labkoff S, Eilenberg K, et al. Trustworthy reuse of health data: a transnational perspective. Int J Med Inf 2013;82:1–9. https://doi.org/10.1016/j.ijmedinf.2012.11.003.
    PubMed
  • 77 Semler SC, Wissing F, Heyder R. German Medical Informatics Initiative. Methods Inf Med 2018;57:e50–6. https://doi.org/10.3414/ME18-03-0003.
    PubMed
  • 78 Sahoo SS, Lhatoo SD, Gupta DK, Cui L, Zhao M, Jayapandian C, et al. Epilepsy and seizure ontology: towards an epilepsy informatics infrastructure for clinical research and patient care. J Am Med Inform Assoc JAMIA 2014;21:82–9. https://doi.org/10.1136/amiajnl-2013-001696.
    PubMed
  • 79 McMurry AJ, Murphy SN, MacFadden D, Weber G, Simons WW, Orechia J, et al. SHRINE: enabling nationally scalable multi-site disease studies. PloS One 2013;8:e55811. https://doi.org/10.1371/journal.pone.0055811.
    PubMed
  • 80 Sinaci AA, Laleci Erturkmen GB. A federated semantic metadata registry framework for enabling interoperability across clinical research and care domains. J Biomed Inform 2013;46:784–94. https://doi.org/10.1016/j.jbi.2013.05.009.
    PubMed
  • 81 Pathak J, Bailey KR, Beebe CE, Bethard S, Carrell DC, Chen PJ, et al. Normalization and standardization of electronic health records for high-throughput phenotyping: the SHARPn consortium. J Am Med Inform Assoc JAMIA 2013;20:e341-348. https://doi.org/10.1136/amiajnl-2013-001939.
    PubMed
  • 82 Liang SF, Taweel A, Miles S, Kovalchuk Y, Spiridou A, Barratt B, et al. Semi automated transformation to OWL formatted files as an approach to data integration. A feasibility study using environmental, disease register and primary care clinical data. Methods Inf Med 2015;54:32–40. https://doi.org/10.3414/ME13-02-0029.
    PubMed
  • 83 Choquet R, Maaroufi M, de Carrara A, Messiaen C, Luigi E, Landais P. A methodology for a minimum data set for rare diseases to support national centers of excellence for healthcare and research. J Am Med Inform Assoc JAMIA 2015;22:76–85. https://doi.org/10.1136/amiajnl-2014-002794.
    PubMed
  • 84 Pang C, Hendriksen D, Dijkstra M, van der Velde KJ, Kuiper J, Hillege HL, et al. BiobankConnect: software to rapidly connect data elements for pooled analysis across biobanks using ontological and lexical indexing. J Am Med Inform Assoc JAMIA 2015;22:65–75. https://doi.org/10.1136/amiajnl-2013-002577.
    PubMed
  • 85 Jiang G, Evans J, Oniki TA, Coyle JF, Bain L, Huff SM, et al. Harmonization of detailed clinical models with clinical study data standards. Methods Inf Med 2015;54:65–74. https://doi.org/10.3414/ME13-02-0019.
    PubMed
  • 86 Huser V, Sastry C, Breymaier M, Idriss A, Cimino JJ. Standardizing data exchange for clinical research protocols and case report forms: An assessment of the suitability of the Clinical Data Interchange Standards Consortium (CDISC) Operational Data Model (ODM). J Biomed Inform 2015;57:88–99. https://doi.org/10.1016/j.jbi.2015.06.023.
    PubMed
  • 87 Wilkinson MD, Dumontier M, Aalbersberg IJJ, Appleton G, Axton M, Baak A, et al. The FAIR Guiding Principles for scientific data management and stewardship. Sci Data 2016;3:160018. https://doi.org/10.1038/sdata.2016.18.
    PubMed
  • 88 Garza M, Del Fiol G, Tenenbaum J, Walden A, Zozus MN. Evaluating common data models for use with a longitudinal community registry. J Biomed Inform 2016;64:333–41. https://doi.org/10.1016/j.jbi.2016.10.016.
    PubMed
  • 89 Hripcsak G, Ryan PB, Duke JD, Shah NH, Park RW, Huser V, et al. Characterizing treatment pathways at scale using the OHDSI network. Proc Natl Acad Sci U S A 2016;113:7329–36. https://doi.org/10.1073/pnas.1510502113.
    PubMed
  • 90 Alonso-Calvo R, Paraiso-Medina S, Perez-Rey D, Alonso-Oset E, van Stiphout R, Yu S, et al. A semantic interoperability approach to support integration of gene expression and clinical data in breast cancer. Comput Biol Med 2017;87:179–86. https://doi.org/10.1016/j.compbiomed.2017.06.005.
    PubMed
  • 91 Yamamoto K, Ota K, Akiya I, Shintani A. A pragmatic method for transforming clinical research data from the research electronic data capture “REDCap” to Clinical Data Interchange Standards Consortium (CDISC) Study Data Tabulation Model (SDTM): Development and evaluation of REDCap2SDTM. J Biomed Inform 2017;70:65–76. https://doi.org/10.1016/j.jbi.2017.05.003.
    PubMed
  • 92 Paris N, Mendis M, Daniel C, Murphy S, Tannier X, Zweigenbaum P. i2b2 implemented over SMART-on-FHIR. AMIA Jt Summits Transl Sci Proc AMIA Jt Summits Transl Sci 2018;2017:369–78.
    PubMed
  • 93 Pedrera-Jiménez M, García-Barrio N, Cruz-Rojo J, Terriza-Torres AI, López-Jiménez EA, Calvo-Boyero F, et al. Obtaining EHR-derived datasets for COVID-19 research within a short time: a flexible methodology based on Detailed Clinical Models. J Biomed Inform 2021;115:103697. https://doi.org/10.1016/j.jbi.2021.103697.
    PubMed
  • 94 Phuong J, Zampino E, Dobbins N, Espinoza J, Meeker D, Spratt H, et al. Extracting Patient-level Social Determinants of Health into the OMOP Common Data Model. AMIA Annu Symp Proc AMIA Symp 2021;2021:989–98.
    PubMed
  • 95 Gruendner J, Deppenwiese N, Folz M, Köhler T, Kroll B, Prokosch H-U, et al. The Architecture of a Feasibility Query Portal for Distributed COVID-19 Fast Healthcare Interoperability Resources (FHIR) Patient Data Repositories: Design and Implementation Study. JMIR Med Inform 2022;10:e36709. https://doi.org/10.2196/36709.
    PubMed
  • 96 Callahan TJ, Stefanski AL, Wyrwa JM, Zeng C, Ostropolets A, Banda JM, et al. Ontologizing health systems data at scale: making translational discovery a reality. NPJ Digit Med 2023;6:89. https://doi.org/10.1038/s41746-023-00830-x.
    PubMed
  • 97 Rusanov A, Weiskopf NG, Wang S, Weng C. Hidden in plain sight: bias towards sick patients when sampling patients with sufficient electronic health record data for research. BMC Med Inform Decis Mak 2014;14:51. https://doi.org/10.1186/1472-6947-14-51.
    PubMed
  • 98 Johnson SG, Speedie S, Simon G, Kumar V, Westra BL. Application of An Ontology for Characterizing Data Quality For a Secondary Use of EHR Data. Appl Clin Inform 2016;7:69–88. https://doi.org/10.4338/ACI-2015-08-RA-0107.
    PubMed
  • 99 Kahn MG, Callahan TJ, Barnard J, Bauck AE, Brown J, Davidson BN, et al. A Harmonized Data Quality Assessment Terminology and Framework for the Secondary Use of Electronic Health Record Data. EGEMS Wash DC 2016;4:1244. https://doi.org/10.13063/2327-9214.1244.
    PubMed
  • 100 Estiri H, Stephens KA, Klann JG, Murphy SN. Exploring completeness in clinical data research networks with DQe-c. J Am Med Inform Assoc JAMIA 2018;25:17–24. https://doi.org/10.1093/jamia/ocx109.
    PubMed
  • 101 Qualls LG, Phillips TA, Hammill BG, Topping J, Louzao DM, Brown JS, et al. Evaluating Foundational Data Quality in the National Patient-Centered Clinical Research Network (PCORnet®). EGEMS Wash DC 2018;6:3. https://doi.org/10.5334/egems.199.
    PubMed
  • 102 Daniel C, Serre P, Orlova N, Bréant S, Paris N, Griffon N. Initializing a hospital-wide data quality program. The AP-HP experience. Comput Methods Programs Biomed 2019;181:104804. https://doi.org/10.1016/j.cmpb.2018.10.016.
    PubMed
  • 103 Bell SK, Delbanco T, Elmore JG, Fitzgerald PS, Fossa A, Harcourt K, et al. Frequency and Types of Patient-Reported Errors in Electronic Health Record Ambulatory Care Notes. JAMA Netw Open 2020;3:e205867. https://doi.org/10.1001/amanetworkopen.2020.5867.
    PubMed
  • 104 Bian J, Lyu T, Loiacono A, Viramontes TM, Lipori G, Guo Y, et al. Assessing the practice of data quality evaluation in a national clinical data research network through a systematic scoping review in the era of real-world data. J Am Med Inform Assoc JAMIA 2020;27:1999–2010. https://doi.org/10.1093/jamia/ocaa245.
    PubMed
  • 105 Mathes T, Rombey T, Kuss O, Pieper D. No inexplicable disagreements between real-world data-based nonrandomized controlled studies and randomized controlled trials were found. J Clin Epidemiol 2021;133:1–13. https://doi.org/10.1016/j.jclinepi.2020.12.019.
    PubMed
  • 106 Blacketer C, Defalco FJ, Ryan PB, Rijnbeek PR. Increasing trust in real-world evidence through evaluation of observational data quality. J Am Med Inform Assoc JAMIA 2021;28:2251–7. https://doi.org/10.1093/jamia/ocab132.
    PubMed
  • 107 Tahar K, Martin T, Mou Y, Verbuecheln R, Graessner H, Krefting D. Rare Diseases in Hospital Information Systems-An Interoperable Methodology for Distributed Data Quality Assessments. Methods Inf Med 2023;62:71–89. https://doi.org/10.1055/a-2006-1018.
    PubMed
  • 108 Syed R, Eden R, Makasi T, Chukwudi I, Mamudu A, Kamalpour M, et al. Digital Health Data Quality Issues: Systematic Review. J Med Internet Res 2023;25:e42615. https://doi.org/10.2196/42615.
    PubMed
  • 109 Lee S, Roh G-H, Kim J-Y, Ho Lee Y, Woo H, Lee S. Effective data quality management for electronic medical record data using SMART DATA. Int J Med Inf 2023;180:105262. https://doi.org/10.1016/j.ijmedinf.2023.105262.
    PubMed
  • 110 Declerck J, Kalra D, Vander Stichele R, Coorevits P. Frameworks, Dimensions, Definitions of Aspects, and Assessment Methods for the Appraisal of Quality of Health Data for Secondary Use: Comprehensive Overview of Reviews. JMIR Med Inform 2024;12:e51560. https://doi.org/10.2196/51560.
    PubMed
  • 111 Eder J, Gottweis H, Zatloukal K. IT solutions for privacy protection in biobanking. Public Health Genomics 2012;15:254–62. https://doi.org/10.1159/000336663.
    PubMed
  • 112 Atreya RV, Smith JC, McCoy AB, Malin B, Miller RA. Reducing patient re-identification risk for laboratory results within research datasets. J Am Med Inform Assoc JAMIA 2013;20:95–101. https://doi.org/10.1136/amiajnl-2012-001026.
    PubMed
  • 113 Kho AN, Cashy JP, Jackson KL, Pah AR, Goel S, Boehnke J, et al. Design and implementation of a privacy preserving electronic health record linkage tool in Chicago. J Am Med Inform Assoc JAMIA 2015;22:1072–80. https://doi.org/10.1093/jamia/ocv038.
    PubMed
  • 114 Harmanci A, Gerstein M. Quantification of private information leakage from phenotype-genotype data: linking attacks. Nat Methods 2016;13:251–6. https://doi.org/10.1038/nmeth.3746.
    PubMed
  • 115 Prasser F, Spengler H, Bild R, Eicher J, Kuhn KA. Privacy-enhancing ETL-processes for biomedical data. Int J Med Inf 2019;126:72–81. https://doi.org/10.1016/j.ijmedinf.2019.03.006.
    PubMed
  • 116 Zhang Z, Yan C, Mesa DA, Sun J, Malin BA. Ensuring electronic medical record simulation through better training, modeling, and evaluation. J Am Med Inform Assoc JAMIA 2020;27:99–108. https://doi.org/10.1093/jamia/ocz161.
    PubMed
  • 117 Paddock S, Abedtash H, Zummo J, Thomas S. Proof-of-concept study: Homomorphically encrypted data can support real-time learning in personalized cancer medicine. BMC Med Inform Decis Mak 2019;19:255. https://doi.org/10.1186/s12911-019-0983-9.
    PubMed
  • 118 Foraker RE, Yu SC, Gupta A, Michelson AP, Pineda Soto JA, Colvin R, et al. Spot the difference: comparing results of analyses from real patient data and synthetic derivatives. JAMIA Open 2020;3:557–66. https://doi.org/10.1093/jamiaopen/ooaa060.
    PubMed
  • 119 Marsolo K, Kiernan D, Toh S, Phua J, Louzao D, Haynes K, et al. Assessing the impact of privacy-preserving record linkage on record overlap and patient demographic and clinical characteristics in PCORnet®, the National Patient-Centered Clinical Research Network. J Am Med Inform Assoc JAMIA 2023;30:447–55. https://doi.org/10.1093/jamia/ocac229.
    PubMed
  • 120 Hripcsak G, Albers DJ. Next-generation phenotyping of electronic health records. J Am Med Inform Assoc JAMIA 2013;20:117–21. https://doi.org/10.1136/amiajnl-2012-001145.
    PubMed
  • 121 Newton KM, Peissig PL, Kho AN, Bielinski SJ, Berg RL, Choudhary V, et al. Validation of electronic medical record-based phenotyping algorithms: results and lessons learned from the eMERGE network. J Am Med Inform Assoc JAMIA 2013;20:e147-154. https://doi.org/10.1136/amiajnl-2012-000896.
    PubMed
  • 122 Kirby JC, Speltz P, Rasmussen LV, Basford M, Gottesman O, Peissig PL, et al. PheKB: a catalog and workflow for creating electronic phenotype algorithms for transportability. J Am Med Inform Assoc JAMIA 2016;23:1046–52. https://doi.org/10.1093/jamia/ocv202.
    PubMed
  • 123 Pivovarov R, Perotte AJ, Grave E, Angiolillo J, Wiggins CH, Elhadad N. Learning probabilistic phenotypes from heterogeneous EHR data. J Biomed Inform 2015;58:156–65. https://doi.org/10.1016/j.jbi.2015.10.001.
    PubMed
  • 124 Halpern Y, Horng S, Choi Y, Sontag D. Electronic medical record phenotyping using the anchor and learn framework. J Am Med Inform Assoc JAMIA 2016;23:731–40. https://doi.org/10.1093/jamia/ocw011.
    PubMed
  • 125 Agarwal V, Podchiyska T, Banda JM, Goel V, Leung TI, Minty EP, et al. Learning statistical models of phenotypes using noisy labeled training data. J Am Med Inform Assoc JAMIA 2016;23:1166–73. https://doi.org/10.1093/jamia/ocw028.
    PubMed
  • 126 Luo L, Li L, Hu J, Wang X, Hou B, Zhang T, et al. A hybrid solution for extracting structured medical information from unstructured data in medical records via a double-reading/entry system. BMC Med Inform Decis Mak 2016;16:114. https://doi.org/10.1186/s12911-016-0357-5.
    PubMed
  • 127 Brisimi TS, Chen R, Mela T, Olshevsky A, Paschalidis IC, Shi W. Federated learning of predictive models from federated Electronic Health Records. Int J Med Inf 2018;112:59–67. https://doi.org/10.1016/j.ijmedinf.2018.01.007.
    PubMed
  • 128 Agniel D, Kohane IS, Weber GM. Biases in electronic health record data due to processes within the healthcare system: retrospective observational study. BMJ 2018;361:k1479. https://doi.org/10.1136/bmj.k1479.
    PubMed
  • 129 Zhang Y, Cai T, Yu S, Cho K, Hong C, Sun J, et al. High-throughput phenotyping with electronic medical record data using a common semi-supervised approach (PheCAP). Nat Protoc 2019;14:3426–44. https://doi.org/10.1038/s41596-019-0227-6.
    PubMed
  • 130 Kashyap M, Seneviratne M, Banda JM, Falconer T, Ryu B, Yoo S, et al. Development and validation of phenotype classifiers across multiple sites in the observational health data sciences and informatics network. J Am Med Inform Assoc JAMIA 2020;27:877–83. https://doi.org/10.1093/jamia/ocaa032.
    PubMed
  • 131 Eaneff S, Obermeyer Z, Butte AJ. The Case for Algorithmic Stewardship for Artificial Intelligence and Machine Learning Technologies. JAMA 2020;324:1397–8. https://doi.org/10.1001/jama.2020.9371.
    PubMed
  • 132 Estiri H, Strasser ZH, Murphy SN. High-throughput phenotyping with temporal sequences. J Am Med Inform Assoc JAMIA 2021;28:772–81. https://doi.org/10.1093/jamia/ocaa288.
    PubMed
  • 133 Bedoya AD, Economou-Zavlanos NJ, Goldstein BA, Young A, Jelovsek JE, O'Brien C, et al. A framework for the oversight and local deployment of safe and high-quality prediction models. J Am Med Inform Assoc JAMIA 2022;29:1631–6. https://doi.org/10.1093/jamia/ocac078.
    PubMed
  • 134 Ahuja Y, Zou Y, Verma A, Buckeridge D, Li Y. MixEHR-Guided: A guided multi-modal topic modeling approach for large-scale automatic phenotyping using the electronic health record. J Biomed Inform 2022;134:104190. https://doi.org/10.1016/j.jbi.2022.104190.
    PubMed
  • 135 Keloth VK, Banda JM, Gurley M, Heider PM, Kennedy G, Liu H, et al. Representing and utilizing clinical textual data for real world studies: An OHDSI approach. J Biomed Inform 2023;142:104343. https://doi.org/10.1016/j.jbi.2023.104343.
    PubMed
  • 136 Hamidi B, Flume PA, Simpson KN, Alekseyenko AV. Not all phenotypes are created equal: covariates of success in e-phenotype specification. J Am Med Inform Assoc JAMIA 2023;30:213–21. https://doi.org/10.1093/jamia/ocac157.
    PubMed
  • 137 Brandt PS, Kho A, Luo Y, Pacheco JA, Walunas TL, Hakonarson H, et al. Characterizing variability of electronic health record-driven phenotype definitions. J Am Med Inform Assoc JAMIA 2023;30:427–37. https://doi.org/10.1093/jamia/ocac235.
    PubMed
  • 138 Huser V, Cimino JJ. Linking ClinicalTrials.gov and PubMed to track results of interventional human clinical trials. PloS One 2013;8:e68409. https://doi.org/10.1371/journal.pone.0068409.
    PubMed
  • 139 Heneberg P. From Excessive Journal Self-Cites to Citation Stacking: Analysis of Journal Self-Citation Kinetics in Search for Journals, Which Boost Their Scientometric Indicators. PloS One 2016;11:e0153730. https://doi.org/10.1371/journal.pone.0153730.
    PubMed
  • 140 Kohane IS, Aronow BJ, Avillach P, Beaulieu-Jones BK, Bellazzi R, Bradford RL, et al. What Every Reader Should Know About Studies Using Electronic Health Record Data but May Be Afraid to Ask. J Med Internet Res 2021;23:e22219. https://doi.org/10.2196/22219.
    PubMed
  • 141 McIntosh LD, Juehne A, Vitale CRH, Liu X, Alcoser R, Lukas JC, et al. Repeat: a framework to assess empirical reproducibility in biomedical research. BMC Med Res Methodol 2017;17:143. https://doi.org/10.1186/s12874-017-0377-6.
    PubMed
  • 142 Slager SL, Weir CR, Kim H, Mostafa J, Del Fiol G. Physicians' perception of alternative displays of clinical research evidence for clinical decision support - A study with case vignettes. J Biomed Inform 2017;71S:S53–9. https://doi.org/10.1016/j.jbi.2017.01.007.
    PubMed
  • 143 Rea S, Pathak J, Savova G, Oniki TA, Westberg L, Beebe CE, et al. Building a robust, scalable and standards-driven infrastructure for secondary use of EHR data: the SHARPn project. J Biomed Inform 2012;45:763–71. https://doi.org/10.1016/j.jbi.2012.01.009.
    PubMed
  • 144 O'Driscoll A, Daugelaite J, Sleator RD. “Big data”, Hadoop and cloud computing in genomics. J Biomed Inform 2013;46:774–81. https://doi.org/10.1016/j.jbi.2013.07.001.
    PubMed
  • 145 Sahoo SS, Jayapandian C, Garg G, Kaffashi F, Chung S, Bozorgi A, et al. Heart beats in the cloud: distributed analysis of electrophysiological “Big Data” using cloud computing for epilepsy clinical research. J Am Med Inform Assoc JAMIA 2014;21:263–71. https://doi.org/10.1136/amiajnl-2013-002156.
    PubMed
  • 146 Bellazzi R, Dagliati A, Sacchi L, Segagni D. Big Data Technologies: New Opportunities for Diabetes Management. J Diabetes Sci Technol 2015;9:1119–25. https://doi.177/1932296815583505.
    PubMed
  • 147 Schüssler-Fiorenza Rose SM, Contrepois K, Moneghetti KJ, Zhou W, Mishra T, Mataraso S, et al. A longitudinal big data approach for precision health. Nat Med 2019;25:792–804. https://doi.org/10.1038/s41591-019-0414-6.
    PubMed
  • 148 Bahmani A, Alavi A, Buergel T, Upadhyayula S, Wang Q, Ananthakrishnan SK, et al. A scalable, secure, and interoperable platform for deep data-driven health management. Nat Commun 2021;12:5757. https://doi.org/10.1038/s41467-021-26040-1.
    PubMed
  • 149 Brat GA, Weber GM, Gehlenborg N, Avillach P, Palmer NP, Chiovato L, et al. International electronic health record-derived COVID-19 clinical course profiles: the 4CE consortium. NPJ Digit Med 2020;3:109. https://doi.org/10.1038/s41746-020-00308-0.
    PubMed
  • 150 Kim SYH, Miller FG. Informed consent for pragmatic trials--the integrated consent model. N Engl J Med 2014;370:769–72. https://doi.org/10.1056/NEJMhle1312508.
    PubMed
  • 151 Sugarman J, Califf RM. Ethics and regulatory complexities for pragmatic clinical trials. JAMA 2014;311:2381–2. https://doi.org/10.1001/jama.2014.4164.
    PubMed
  • 152 Gray EA, Thorpe JH. Comparative effectiveness research and big data: balancing potential with legal and ethical considerations. J Comp Eff Res 2015;4:61–74. https://doi.org/10.2217/cer.14.51.
    PubMed
  • 153 Kim H, Bell E, Kim J, Sitapati A, Ramsdell J, Farcas C, et al. iCONCUR: informed consent for clinical data and bio-sample use for research. J Am Med Inform Assoc JAMIA 2017;24:380–7. https://doi.org/10.1093/jamia/ocw115.
    PubMed
  • 154 Lea NC, Nicholls J, Fitzpatrick NK. Between Scylla and Charybdis: Charting the Wicked Problem of Reusing Health Data for Clinical Research Informatics. Yearb Med Inform 2018;27:170–6. https://doi.rg/10.1055/s-0038-1641219.
    PubMed
  • 155 Chandler R, Brady KT, Jerome RN, Eder M, Rothwell E, Brownley KA, et al. Broad-scale informed consent: A survey of the CTSA landscape. J Clin Transl Sci 2019;3:253–60. https://doi.org/10.1017/cts.2019.397.
    PubMed
  • 156 Ferretti A, Ienca M, Sheehan M, Blasimme A, Dove ES, Farsides B, et al. Ethics review of big data research: What should stay and what should be reformed? BMC Med Ethics 2021;22:51. https://doi.org/10.1186/s12910-021-00616-4.
    PubMed
  • 157 Christofidou M, Lea N, Coorevits P. A Literature Review on the GDPR, COVID-19 and the Ethical Considerations of Data Protection During a Time of Crisis. Yearb Med Inform 2021;30:226–32. https://doi.rg/10.1055/s-0041-1726512.
    PubMed
  • 158 Kargl M, Plass M, Müller H. A Literature Review on Ethics for AI in Biomedical Research and Biobanking. Yearb Med Inform 2022;31:152–60. https://doi.rg/10.1055/s-0042-1742516.
    PubMed
  • 159 Larson DB, Harvey H, Rubin DL, Irani N, Tse JR, Langlotz CP. Regulatory Frameworks for Development and Evaluation of Artificial Intelligence-Based Diagnostic Imaging Algorithms: Summary and Recommendations. J Am Coll Radiol JACR 2021;18:413–24. https://doi.org/10.1016/j.jacr.2020.09.060.
    PubMed
  • 160 Solomonides AE, Koski E, Atabaki SM, Weinberg S, McGreevey JD, Kannry JL, et al. Defining AMIA's artificial intelligence principles. J Am Med Inform Assoc JAMIA 2022;29:585–91. https://doi.org/10.1093/jamia/ocac006.
    PubMed
  • 161 Ohno-Machado L. NIH's Big Data to Knowledge initiative and the advancement of biomedical informatics. J Am Med Inform Assoc JAMIA 2014;21:193. https://doi.org/10.1136/amiajnl-2014-002666.
    PubMed
  • 162 Holmes JH, Elliott TE, Brown JS, Raebel MA, Davidson A, Nelson AF, et al. Clinical research data warehouse governance for distributed research networks in the USA: a systematic review of the literature. J Am Med Inform Assoc JAMIA 2014;21:730–6. https://doi.org/10.1136/amiajnl-2013-002370.
    PubMed
  • 163 Wilcox A, Randhawa G, Embi P, Cao H, Kuperman GJ. Sustainability considerations for health research and analytic data infrastructures. EGEMS Wash DC 2014;2:1113. https://doi.org/10.13063/2327-9214.1113.
    PubMed
  • 164 Unertl KM, Schaefbauer CL, Campbell TR, Senteio C, Siek KA, Bakken S, et al. Integrating community-based participatory research and informatics approaches to improve the engagement and health of underserved populations. J Am Med Inform Assoc JAMIA 2016;23:60–73. https://doi.org/10.1093/jamia/ocv094.
    PubMed
  • 165 Longo DL, Drazen JM. Data Sharing. N Engl J Med 2016;374:276–7. https://doi.org/10.1056/NEJMe1516564.
    PubMed
  • 166 Wiley LK, Tarczy-Hornoch P, Denny JC, Freimuth RR, Overby CL, Shah N, et al. Harnessing next-generation informatics for personalizing medicine: a report from AMIA's 2014 Health Policy Invitational Meeting. J Am Med Inform Assoc JAMIA 2016;23:413–9. https://doi.org/10.1093/jamia/ocv111.
    PubMed
  • 167 McCray AT, Glaser J, Koppel R, Langlotz CP, Silverstein J. Health IT vendors and the academic community: The 2014 ACMI debate. J Biomed Inform 2016;60:365–75. https://doi.org/10.1016/j.jbi.2016.03.003.
    PubMed
  • 168 Spencer K, Sanders C, Whitley EA, Lund D, Kaye J, Dixon WG. Patient Perspectives on Sharing Anonymized Personal Health Data Using a Digital System for Dynamic Consent and Research Feedback: A Qualitative Study. J Med Internet Res 2016;18:e66. https://doi.org/10.2196/jmir.5011.
    PubMed
  • 169 Valenta AL, Meagher EA, Tachinardi U, Starren J. Core informatics competencies for clinical and translational scientists: what do our customers and collaborators need to know? J Am Med Inform Assoc JAMIA 2016;23:835–9. https://doi.org/10.1093/jamia/ocw047.
    PubMed
  • 170 Snyder CF, Smith KC, Bantug ET, Tolbert EE, Blackford AL, Brundage MD, et al. What do these scores mean? Presenting patient-reported outcomes data to patients and clinicians to improve interpretability. Cancer 2017;123:1848–59. https://doi.org/10.1002/cncr.30530.
    PubMed
  • 171 Adler-Milstein J, Embi PJ, Middleton B, Sarkar IN, Smith J. Crossing the health IT chasm: considerations and policy recommendations to overcome current challenges and enable value-based care. J Am Med Inform Assoc JAMIA 2017;24:1036–43. https://doi.org/10.1093/jamia/ocx017.
    PubMed
  • 172 Richesson RL, Green BB, Laws R, Puro J, Kahn MG, Bauck A, et al. Pragmatic (trial) informatics: a perspective from the NIH Health Care Systems Research Collaboratory. J Am Med Inform Assoc JAMIA 2017;24:996–1001. https://doi.org/10.1093/jamia/ocx016.
    PubMed
  • 173 Sydes MR, Ashby D. Data Authorship as an Incentive to Data Sharing. N Engl J Med 2017;377:402. https://doi.org/10.1056/NEJMc1707245.
    PubMed
  • 174 Sanchez-Pinto LN, Mosa ASM, Fultz-Hollis K, Tachinardi U, Barnett WK, Embi PJ. The Emerging Role of the Chief Research Informatics Officer in Academic Health Centers. Appl Clin Inform 2017;8:845–53. https://doi.org/10.4338/ACI-2017-04-RA-0062.
    PubMed
  • 175 Franklin P, Chenok K, Lavalee D, Love R, Paxton L, Segal C, et al. Framework To Guide The Collection And Use Of Patient-Reported Outcome Measures In The Learning Healthcare System. EGEMS Wash DC 2017;5:17. https://doi.org/10.5334/egems.227.
    PubMed
  • 176 Beier K, Schweda M, Schicktanz S. Taking patient involvement seriously: a critical ethical analysis of participatory approaches in data-intensive medical research. BMC Med Inform Decis Mak 2019;19:90. https://doi.org/10.1186/s12911-019-0799-7.
    PubMed
  • 177 Kalra D, Stroetmann V, Sundgren M, Dupont D, Schlünder I, Thienpont G, et al. The European Institute for Innovation through Health Data. Learn Health Syst 2017;1:e10008. https://doi.org/10.1002/lrh2.10008.
    PubMed
  • 178 Embi PJ, Richesson R, Tenenbaum J, Kannry J, Friedman C, Sarkar IN, et al. Reimagining the research-practice relationship: policy recommendations for informatics-enabled evidence-generation across the US health system. JAMIA Open 2019;2:2–9. https://doi.org/10.1093/jamiaopen/ooy056.
    PubMed
  • 179 Zayas-Cabán T, Chaney KJ, Rucker DW. National health information technology priorities for research: A policy and development agenda. J Am Med Inform Assoc JAMIA 2020;27:652–7. https://doi.org/10.1093/jamia/ocaa008.
    PubMed
  • 180 Byrne LM, Cook SK, Kennedy N, Russell M, Jerome RN, Tan J, et al. Opening doors to clinical trial participation among Hispanics: Lessons learned from the Spanish translation of ResearchMatch. J Clin Transl Sci 2020;5:e46. https://doi.org/10.1017/cts.2020.539.
    PubMed
  • 181 Zayas-Cabán T, Abernethy AP, Brennan PF, Devaney S, Kerlavage AR, Ramoni R, et al. Leveraging the health information technology infrastructure to advance federal research priorities. J Am Med Inform Assoc JAMIA 2020;27:647–51. https://doi.org/10.1093/jamia/ocaa011.
    PubMed
  • 182 Zenker S, Strech D, Ihrig K, Jahns R, Müller G, Schickhardt C, et al. Data protection-compliant broad consent for secondary use of health care data and human biosamples for (bio)medical research: Towards a new German national standard. J Biomed Inform 2022;131:104096. https://doi.org/10.1016/j.jbi.2022.104096.
    PubMed
  • 183 Peters AE, Ogunniyi MO, Hegde SM, Bianco C, Ghafghazi S, Hernandez AF, et al. A multicenter program for electronic health record screening for patients with heart failure with preserved ejection fraction: Lessons from the DELIVER-EHR initiative. Contemp Clin Trials 2022;121:106924. https://doi.org/10.1016/j.cct.2022.106924.
    PubMed
  • 184 Maurud S, Henni SH, Moen A. Health Equity in Clinical Research Informatics. Yearb Med Inform 2023;32:138–45. https://doi.rg/10.1055/s-0043-1768720.
    PubMed
  • 185 Alberto IRI, Alberto NRI, Ghosh AK, Jain B, Jayakumar S, Martinez-Martin N, et al. The impact of commercial health datasets on medical research and health-care algorithms. Lancet Digit Health 2023;5:e288–94. https://doi.org/10.1016/S2589-7500(23)00025-0.
    PubMed
  • 186 Russell ES, Aubrun E, Moga DC, Guedes S, Camelo Castillo W, Hardy JR, et al. FDA draft guidance to improve clinical trial diversity: Opportunities for pharmacoepidemiology. J Clin Transl Sci 2023;7:e101. https://doi.org/10.1017/cts.2023.515.
    PubMed
  • 187 Royce TJ, Zhao Y, Ryals CA. Improving Diversity in Clinical Trials by Using Real-world Data to Define Eligibility Criteria. JAMA Oncol 2023;9:455–6. https://doi.org/10.1001/jamaoncol.2022.7170.
    PubMed
  • 188 Patel VN, Kaelber DC. Using aggregated, de-identified electronic health record data for multivariate pharmacosurveillance: a case study of azathioprine. J Biomed Inform 2014;52:36–42. https://doi.org/10.1016/j.jbi.2013.10.009.
    PubMed
  • 189 Trifirò G, Coloma PM, Rijnbeek PR, Romio S, Mosseveld B, Weibel D, et al. Combining multiple healthcare databases for postmarketing drug and vaccine safety surveillance: why and how? J Intern Med 2014;275:551–61. https://doi.org/10.1111/joim.12159.
    PubMed
  • 190 Koutkias VG, Jaulent M-C. Computational approaches for pharmacovigilance signal detection: toward integrated and semantically-enriched frameworks. Drug Saf 2015;38:219–32. https://doi.org/10.1007/s40264-015-0278-8.
    PubMed
  • 191 Marsolo K, Margolis PA, Forrest CB, Colletti RB, Hutton JJ. A Digital Architecture for a Network-Based Learning Health System: Integrating Chronic Care Management, Quality Improvement, and Research. EGEMS Wash DC 2015;3:1168. https://doi.org/10.13063/2327-9214.1168.
    PubMed
  • 192 Harle CA, Lipori G, Hurley RW. Collecting, Integrating, and Disseminating Patient-Reported Outcomes for Research in a Learning Healthcare System. EGEMS Wash DC 2016;4:1240. https://doi.org/10.13063/2327-9214.1240.
    PubMed
  • 193 Suchard MA, Schuemie MJ, Krumholz HM, You SC, Chen R, Pratt N, et al. Comprehensive comparative effectiveness and safety of first-line antihypertensive drug classes: a systematic, multinational, large-scale analysis. Lancet Lond Engl 2019;394:1816–26. https://doi.org/10.1016/S0140-6736(19)32317-7.
    PubMed
  • 194 Platt JE, Raj M, Wienroth M. An Analysis of the Learning Health System in Its First Decade in Practice: Scoping Review. J Med Internet Res 2020;22:e17026. https://doi.org/10.2196/17026.
    PubMed
  • 195 Geva A, Stedman JP, Manzi SF, Lin C, Savova GK, Avillach P, et al. Adverse drug event presentation and tracking (ADEPT): semiautomated, high throughput pharmacovigilance using real-world data. JAMIA Open 2020;3:413–21. https://doi.org/10.1093/jamiaopen/ooaa031.
    PubMed
  • 196 Hartley DM, Seid M. Collaborative learning health systems: Science and practice. Learn Health Syst 2021;5:e10286. https://doi.org/10.1002/lrh2.10286.
    PubMed
  • 197 Xu H, Aldrich MC, Chen Q, Liu H, Peterson NB, Dai Q, et al. Validating drug repurposing signals using electronic health records: a case study of metformin associated with reduced cancer mortality. J Am Med Inform Assoc JAMIA 2015;22:179–91. https://doi.org/10.1136/amiajnl-2014-002649.
    PubMed
  • 198 Park Y, Jackson GP, Foreman MA, Gruen D, Hu J, Das AK. Evaluating artificial intelligence in medicine: phases of clinical research. JAMIA Open 2020;3:326–31. https://doi.org/10.1093/jamiaopen/ooaa033.
    PubMed
  • 199 Mullen CG, Houlihan JY, Stroo M, Deeter CE, Freel SA, Padget AM, et al. Leveraging retooled clinical research infrastructure for Clinical Research Management System implementation at a large Academic Medical Center. J Clin Transl Sci 2023;7:e127. https://doi.org/10.1017/cts.2023.550.
    PubMed
  • 200 Kwee SA, Wong LL, Ludema C, Deng CK, Taira D, Seto T, et al. Target Trial Emulation: A Design Tool for Cancer Clinical Trials. JCO Clin Cancer Inform 2023;7:e2200140. https://doi.org/10.1200/CCI.22.00140.
    PubMed
  • 201 Wang SV, Schneeweiss S, RCT-DUPLICATE Initiative, Franklin JM, Desai RJ, Feldman W, et al. Emulation of Randomized Clinical Trials With Nonrandomized Database Analyses: Results of 32 Clinical Trials. JAMA 2023;329:1376–85. https://doi.org/10.1001/jama.2023.4221.
    PubMed
  • 202 Ganzinger M, Blumenstock M, Fürstberger A, Greulich L, Kestler HA, Marschollek M, et al. Federated electronic data capture (fEDC): Architecture and prototype. J Biomed Inform 2023;138:104280. https://doi.org/10.1016/j.jbi.2023.104280.
    PubMed
  • 203 Gama F, Tyskbo D, Nygren J, Barlow J, Reed J, Svedberg P. Implementation Frameworks for Artificial Intelligence Translation Into Health Care Practice: Scoping Review. J Med Internet Res 2022;24:e32215. https://doi.org/10.2196/32215.
    PubMed
  • 204 Crossnohere NL, Elsaid M, Paskett J, Bose-Brill S, Bridges JFP. Guidelines for Artificial Intelligence in Medicine: Literature Review and Content Analysis of Frameworks. J Med Internet Res 2022;24:e36823. https://doi.org/10.2196/36823.
    PubMed
  • 205 Cruz Rivera S, Liu X, Chan A-W, Denniston AK, Calvert MJ, SPIRIT-AI and CONSORT-AI Working Group, et al. Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension. Nat Med 2020;26:1351–63. https://doi.org/10.1038/s41591-020-1037-7.
    PubMed