Towards LLM-based fully automated reporting of nuclear medicine examinations at the example of renal scans

Ziel/Aim: Renogram activity curves from scintigraphy data are essential for assessing renal function and identifying pathologies. Although software solutions can generate renogram curves from scintigraphy data, report writing remains manual. Large language models (LLMs) could streamline data extraction and automate report writing, enhancing efficiency in clinical workflows. This study explores the feasibility of using LLMs for automated report generation, starting with textbook cases and expanding to clinical routine scans.

Methodik/Methods: Fourteen cases from a standard textbook (O'Malley, J.P., Ziessman, H.A., 2020) were used, each representing e.g. normal function, impaired function, or obstruction. Both the LLM GPT-4 and human readers labeled each case. In a second step, we deployed Meta's Llama 3.2 11B LLM for interpreting clinical renal scans. To improve diagnostic accuracy, we created an LLM-optimized workflow for automated, comprehensive report generation.

Ergebnisse/Results: In textbook cases, human readers (n=5) achieved 88.6% accuracy and a weighted precision of 92.1%, with a Fleiss Kappa of 0.690, indicating substantial agreement. In contrast, GPT-4 (n=10 revisions) showed 41.4% accuracy and 78.9% weighted precision, with a Fleiss Kappa of 0.425, indicating moderate agreement. A chi-square test showed significant differences between human and GPT-4 classifications (p<0.001). Initial assessments of Meta’s Llama 3.2 11B for routine scans suggest it can accurately extract and report diagnostic information.

Schlussfolgerungen/Conclusions: Human readers significantly outperformed the LLM in categorizing renogram-based scintigraphy data. However, initial tests with clinical data indicate potential for automated report writing. Further studies with larger, real-world datasets are needed to validate these findings.

Publication History

Article published online:
12 March 2025