Comparative Efficacy of ChatGPT and Gemini in Addressing Patient Queries on Gonarthrosis and Total Knee Arthroplasty: A Randomized Controlled Trial

Abstract

The emergence of artificial intelligence (AI) in health care has created novel opportunities for enhancing patient education and alleviating anxiety. This study seeks to evaluate the effectiveness of two leading AI platforms, ChatGPT and Gemini, in delivering accurate and satisfactory responses to patients with gonarthrosis, considering total knee arthroplasty (TKA). A prospective, randomized controlled trial was conducted involving 100 patients diagnosed with gonarthrosis and indicated for TKA. Each patient posed five questions regarding the surgery and postoperative rehabilitation to both ChatGPT and Gemini. Responses were evaluated by two blinded orthopaedic specialists on a 10-point scale for accuracy and patient satisfaction. Patients additionally evaluated their satisfaction with each response using a 10-point scale. The main outcome measures consisted of the average accuracy scores assessed by specialists and the average satisfaction scores reported by patients. Statistical analysis revealed significant differences between ChatGPT and Gemini in both accuracy and patient satisfaction (p < 0.001). ChatGPT demonstrated better performance with a mean accuracy score of 8.7 ± 0.9 compared with Gemini's 7.2 ± 1.1. Patient satisfaction scores aligned with expert evaluations, with ChatGPT achieving a mean satisfaction score of 8.9 ± 0.8 versus Gemini's 7.5 ± 1.2. Notably, ChatGPT excelled in providing comprehensive explanations of surgical procedures (mean score: 9.2 ± 0.7) and postoperative care (9.1 ± 0.8), whereas Gemini performed better in offering concise summaries of recovery timelines (8.4 ± 0.9). This study demonstrates that ChatGPT offers more accurate and satisfactory responses to patient queries regarding gonarthrosis and TKA compared with Gemini. The findings suggest that AI platforms, particularly ChatGPT, can serve as valuable tools in augmenting patient education and potentially reducing preoperative anxiety. Future studies should investigate the incorporation of AI-assisted information delivery into clinical practice and its long-term effects on patient outcomes.

Keywords

artificial intelligence - total knee arthroplasty - patient education - ChatGPT - Gemini - gonarthrosis

• References

• 1 Al Kuwaiti A, Nazer K, Al-Reedy A. et al. A review of the role of artificial intelligence in healthcare. J Pers Med 2023; 13 (06) 951
• 2 Farhadi F, Barnes MR, Sugito HR, Sin JM, Henderson ER, Levy JJ. Applications of artificial intelligence in orthopaedic surgery. Front Med Technol 2022; 4: 995526
• 3 Dave M, Patel N. Artificial intelligence in healthcare and education. Br Dent J 2023; 234 (10) 761-764
• 4 Tam TYC, Sivarajkumar S, Kapoor S. et al. A framework for human evaluation of large language models in healthcare derived from literature review. NPJ Digit Med 2024; 7 (01) 258
• 5 Wang L, Wan Z, Ni C. et al. A systematic review of ChatGPT and other conversational large language models in healthcare. medRxiv 2024 2024.04.26.24306390
• 6 Taylor IV WL, Cheng R, Weinblatt AI, Bergstein V, Long WJ. An artificial intelligence Chatbot is an accurate and useful online patient resource prior to total knee arthroplasty. J Arthroplasty 2024; 39 (8S1): S358-S362
• 7 Hawker GA, Bohm E, Dunbar MJ. et al; BEST-Knee Study Team. Patient appropriateness for total knee arthroplasty and predicted probability of a good outcome. RMD Open 2023; 9 (02) e002808
• 8 Moyer R, Ikert K, Long K, Marsh J. The value of preoperative exercise and education for patients undergoing total hip and knee arthroplasty: a systematic review and meta-analysis. JBJS Rev 2017; 5 (12) e2
• 9 Walker HL, Ghani S, Kuemmerli C. et al. Reliability of medical information provided by ChatGPT: assessment against clinical guidelines and patient information quality instrument. J Med Internet Res 2023; 25: e47479
• 10 Zhou YY, Zhang BK, Ran TF. et al. Education level has an effect on the recovery of total knee arthroplasty: a retrospective study. BMC Musculoskelet Disord 2022; 23 (01) 1072
• 11 Ledziński Ł, Grześk G. Artificial intelligence technologies in cardiology. J Cardiovasc Dev Dis 2023; 10 (05) 202
• 12 Wang R, Wang S, Duan N, Wang Q. From patient-controlled analgesia to artificial intelligence-assisted patient-controlled analgesia: practices and perspectives. Front Med (Lausanne) 2020; 7: 145
• 13 Ruksakulpiwat S, Thorngthip S, Niyomyart A. et al. A systematic review of the application of artificial intelligence in nursing care: where are we, and what's next?. J Multidiscip Healthc 2024; 17: 1603-1616
• 14 Morya VK, Lee HW, Shahid H. et al. Application of ChatGPT for orthopedic surgeries and patient care. Clin Orthop Surg 2024; 16 (03) 347-356
• 15 Boima V, Doku A, Agyekum F, Tuglo LS, Agyemang C. Effectiveness of digital health interventions on blood pressure control, lifestyle behaviours and adherence to medication in patients with hypertension in low-income and middle-income countries: a systematic review and meta-analysis of randomised controlled trials. EClinicalMedicine 2024; 69: 102432
• 16 Lisacek-Kiosoglous AB, Powling AS, Fontalis A, Gabr A, Mazomenos E, Haddad FS. Artificial intelligence in orthopaedic surgery. Bone Joint Res 2023; 12 (07) 447-454