Download PDF
Endoscopy 2025; 57(03): 262-268
DOI: 10.1055/a-2388-6084
Innovations and brief communications

The role of generative language systems in increasing patient awareness of colon cancer screening

Marcello Maida 
1   Department of Medicine and Surgery, University of Enna “Kore,” Enna, Italy
,
Daryl Ramai
2   Division of Gastroenterology and Hepatology, University of Utah Health, Salt Lake City, Utah, USA
,
Yuichi Mori 
3   Clinical Effectiveness Research Group, University of Oslo, Oslo, Norway
4   Digestive Disease Center, Showa University Northern Yokohama Hospital, Yokohama, Japan
,
Mário Dinis-Ribeiro
5   Porto Comprehensive Cancer Center & RISE@CI-IPO, University of Porto, Porto, Portugal
6   Gastroenterology Department, Portuguese Institute of Oncology of Porto, Porto, Portugal
,
Antonio Facciorusso 
7   Gastroenterology Unit, Department of Medical Sciences, University of Foggia, Foggia, Italy
,
Cesare Hassan
8   Endoscopy Unit, Humanitas Clinical and Research Hospital, IRCCS, Rozzano, Italy
9   Department of Biomedical Sciences, Humanitas University, Pieve Emanuele, Milan, Italy
,
and the AI-CORE (Artificial Intelligence COlorectal cancer Research) Working Group
› Author Affiliations
› Further Information
Also available at
   Permissions and Reprints


Referred to by:

Author commentary on Marcello Maida et al.Endoscopy 2025; 57(03): v10-v10
DOI: 10.1055/a-2410-5818

Referred to by:

A needed (r)evolution to improve colorectal cancer screening: binding patient-centered/shared decision-making medicine with large language models in an emerging knowledge societyEndoscopy 2025; 57(03): 269-271
DOI: 10.1055/a-2445-6431

Abstract

Background This study aimed to evaluate the effectiveness of ChatGPT (Chat Generative Pretrained Transformer) in answering patientsʼ questions about colorectal cancer (CRC) screening, with the ultimate goal of enhancing patients' awareness and adherence to national screening programs.

Methods 15 questions on CRC screening were posed to ChatGPT4. The answers were rated by 20 gastroenterology experts and 20 nonexperts in three domains (accuracy, completeness, and comprehensibility), and by 100 patients in three dichotomic domains (completeness, comprehensibility, and trustability).

Results According to expert rating, the mean (SD) accuracy score was 4.8 (1.1), on a scale ranging from 1 to 6. The mean (SD) scores for completeness and comprehensibility were 2.1 (0.7) and 2.8 (0.4), respectively, on scales ranging from 1 to 3. Overall, the mean (SD) accuracy (4.8 [1.1] vs. 5.6 [0.7]; P < 0.001) and completeness scores (2.1 [0.7] vs. 2.7 [0.4]; P < 0.001) were significantly lower for the experts than for the nonexperts, while comprehensibility was comparable among the two groups (2.8 [0.4] vs. 2.8 [0.3]; P = 0.55). Patients rated all questions as complete, comprehensible, and trustable in between 97 % and 100 % of cases.

Conclusions ChatGPT shows good performance, with the potential to enhance awareness about CRC and improve screening outcomes. Generative language systems may be further improved after proper training in accordance with scientific evidence and current guidelines.

Supplementary Material



Publication History

Received: 14 March 2024

Accepted after revision: 14 August 2024

Accepted Manuscript online:
14 August 2024

Article published online:
23 October 2024

© 2024. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 Fitzmaurice C, Dicker D, Pain A. et al. Global burden of disease cancer collaboration. The global burden of cancer 2013. JAMA Oncol 2015; 1: 505-527
    CrossrefPubMedSearch in Google Scholar
  • 2 Løberg M, Kalager M, Holme Ø. et al. Long-term colorectal-cancer mortality after adenoma removal. NEJM 2014; 371: 799-807
    CrossrefPubMedSearch in Google Scholar
  • 3 Bretthauer M, Løberg M, Wieszczy P. et al. Effect of colonoscopy screening on risks of colorectal cancer and related death. NEJM 2022; 387: 1547-1556
    CrossrefPubMedSearch in Google Scholar
  • 4 Hewitson P, Glasziou P, Irwig L. et al. Screening for colorectal cancer using the faecal occult blood test, Hemoccult. Cochrane Database Syst Rev 2007; 2007: CD001216
    PubMedSearch in Google Scholar
  • 5 Tsipa A, O'Connor DB, Branley-Bell D. et al. Promoting colorectal cancer screening: a systematic review and meta-analysis of randomised controlled trials of interventions to increase uptake. Health Psychol Rev 2021; 15: 371-394
    CrossrefPubMedSearch in Google Scholar
  • 6 Hahn EE, Baecker A, Shen E. et al. A patient portal-based commitment device to improve adherence with screening for colorectal cancer: a retrospective observational study. J Gen Intern Med 2021; 36: 952-960
    CrossrefPubMedSearch in Google Scholar
  • 7 Goshgarian G, Sorourdi C, May FP. et al. Effect of patient portal messaging before mailing fecal immunochemical test kit on colorectal cancer screening rates: a randomized clinical trial. JAMA Netw Open 2022; 5: e2146863
    CrossrefPubMedSearch in Google Scholar
  • 8 Klabunde C, Blom J, Bulliard JL. et al. Participation rates for organized colorectal cancer screening programmes: an international comparison. J Med Screen 2015; 22: 119-126
    CrossrefPubMedSearch in Google Scholar
  • 9 McNamara D, Leen R, Seng-Lee C. et al. Sustained participation, colonoscopy uptake and adenoma detection rates over two rounds of the Tallaght-Trinity College colorectal cancer screening programme with the faecal immunological test. Eur J Gastroenterol Hepatol 2014; 26: 1415-1421
    CrossrefPubMedSearch in Google Scholar
  • 10 Kapidzic A, Grobbee EJ, Hol L. et al. Attendance and yield over three rounds of population-based fecal immuno- chemical test screening. Am J Gastroenterol 2014; 109: 1257-1264
    CrossrefPubMedSearch in Google Scholar
  • 11 OpenAI. (2023). ChatGPT (Mar 14 version). Available at (Accessed 5 February 2024): https://chat.openai.com
    PubMed
  • 12 Pugliese N, Wai-Sun Wong V, Schattenberg JM. et al. Accuracy, reliability, and comprehensibility of ChatGPT-generated medical responses for patients with nonalcoholic fatty liver disease. Clin Gastroenterol Hepatol 2024; 22: 886-889.e5
    CrossrefPubMedSearch in Google Scholar
  • 13 Du RC, Liu X, Lai YK. et al. Exploring the performance of ChatGPT on acute pancreatitis-related questions. J Transl Med 2024; 22: 527
    CrossrefPubMedSearch in Google Scholar
  • 14 Lai Y, Liao F, Zhao J. et al. Exploring the capacities of ChatGPT: A comprehensive evaluation of its accuracy and repeatability in addressing helicobacter pylori-related queries. Helicobacter 2024; 29: e13078
    CrossrefPubMedSearch in Google Scholar
  • 15 Lee TC, Staller K, Botoman V. et al. ChatGPT answers common patient questions about colonoscopy. Gastroenterology 2023; 165: 509-511.e7
    CrossrefPubMedSearch in Google Scholar
  • 16 Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika 1951; 16: 297-334
    CrossrefPubMedSearch in Google Scholar