Rofo 2022; 194(08): 852-861
DOI: 10.1055/a-1752-1038
Urogenital Tract

Inter-Reader Variability Using PI-RADS v2 Versus PI-RADS v2.1: Most New Disagreement Stems from Scores 1 and 2

Vergleich der Interrater-Reliabilität von PI-RADS v2 und PI-RADS v2.1: Neue Differenzen entspringen meistens den Bewertungen 1 und 2
1   Department of Radiology, Charite University Hospital Berlin, Germany
,
Matthias Haas
1   Department of Radiology, Charite University Hospital Berlin, Germany
,
Alexander Baur
1   Department of Radiology, Charite University Hospital Berlin, Germany
,
Frank Konietschke
2   Department of Biometry and Clinical Epidemiology, Charite University Hospital Berlin, Germany
,
Akash Roy
3   Biostatistics and Bioinformatics, Duke University School of Medicine, DURHAM, United States
,
Charlie Alexander Hamm
1   Department of Radiology, Charite University Hospital Berlin, Germany
,
Madhuri Monique Rudolph
1   Department of Radiology, Charite University Hospital Berlin, Germany
,
1   Department of Radiology, Charite University Hospital Berlin, Germany
,
Bernd Hamm
1   Department of Radiology, Charite University Hospital Berlin, Germany
,
Hannes Cash
4   Department of Urology, Charite University Hospital Berlin, Germany
,
Patrick Asbach
1   Department of Radiology, Charite University Hospital Berlin, Germany
,
Tobias Penzkofer
1   Department of Radiology, Charite University Hospital Berlin, Germany
› Author Affiliations

Abstract

Purpose To analyze possible differences in the inter-reader variability between PI-RADS version 2 (v2) and version 2.1 (v2.1) for the classification of prostate lesions using multiparametric MRI (mpMRI) of the prostate.

Methods In this retrospective and randomized study, 239 annotated and histopathologically correlated prostate lesions (104 positive and 135 negative for prostate cancer) were rated twice by three experienced uroradiologists using PI-RADS v2 and v2.1 with an interval of at least two months between readings. Results were tabulated across readers and reading timepoints and inter-reader variability was determined using Fleiss’ kappa (κ). Thereafter, an additional analysis of the data was performed in which PI-RADS scores 1 and 2 were combined, as they have the same clinical consequences.

Results PI-PI-RADS v2.1 showed better inter-reader agreement in the peripheral zone (PZ), but poorer inter-reader agreement in the transition zone (TZ) (PZ: κ = 0.63 vs. κ = 0.58; TZ: κ = 0.47 vs. κ = 0.57). When PI-RADS scores 1 and 2 were combined, the use of PI-RADS v2.1 resulted in almost perfect inter-reader agreement in the PZ and substantial agreement in the TZ (PZ: κ = 0.81; TZ: κ = 0.80).

Conclusion PI-RADS v2.1 improves inter-reader agreement in the PZ. New differences in inter-reader agreement were mainly the result of the assignment of PI-RADS v2.1 scores 1 and 2 to lesions in the TZ. Combining scores 1 and 2 improved inter-reader agreement both in the TZ and in the PZ, indicating that refined definitions may be warranted for these PI-RADS scores.

Key Points:

  • PI-RADSv2.1 improves inter-reader agreement in the PZ but not in the TZ.

  • New differences derived from PI-RADSv2.1 scores 1 and 2 in the TZ.

  • Combined PI-RADSv2.1 scores of 1 and 2 yielded better inter-reader agreement.

  • PI-RADSv2.1 appears to provide more precise description of lesions in the PZ.

  • Improved inter-reader agreement in the PZ stresses the importance of appropriate lexicon description.

Citation Format

  • Beetz N, Haas M, Baur A et al. Inter-Reader Variability Using PI-RADS v2 Versus PI-RADS v2.1: Most New Disagreement Stems from Scores 1 and 2. Fortschr Röntgenstr 2022; 194: 852 – 861

Zusammenfassung

Ziel Analyse der möglichen Differenzen in der Interrater-Reliabilität zwischen PI-RADS Version 2 (v2) und Version 2.1 (v2.1) für die Klassifizierung von Prostata-Läsionen in der multiparametrischen MRT (mpMRT) der Prostata.

Methoden In dieser retrospektiven und randomisierten Studie wurden 239 bereits annotierte und histopathologisch korrelierte Läsionen der Prostata (104 positiv und 135 negativ für Prostatakrebs) zweimal von radiologisch-fachärztlichen Experten für uroradiologische Bildgebung bewertet: zuerst nach PI-RADS v2 und nach einem zeitlichen Intervall von mindestens 2 Monaten anhand der aktualisierten Version PI-RADS v2.1. Die Ergebnisse der Bewerter und Bewertungszeitpunkte wurden kreuztabelliert und die Interrater-Reliabilität wurde mittels Fleiss’ kappa (κ) berechnet. In einer weiteren Analyse wurden die beiden Wertungen mit derselben klinischen Konsequenz PI-RADS-Score 1 und 2 kombiniert und deren Interrater-Reliabilität erneut getestet.

Ergebnisse PI-RADS v2.1 zeigte eine bessere Interrater-Reliabilität in der peripheren Zone (PZ), aber eine schlechtere Interrater-Reliabilität in der Transitionszone (TZ) (PZ: κ = 0,63 vs. κ = 0,58; TZ: κ = 0,47 vs. κ = 0,57). Die Kombination der PI-RADS-Bewertungen 1 und 2 bei PI-RADS v2.1 resultierte in einer nahezu perfekten Interrater-Reliabilität in der PZ und in einer substantiellen Interrater-Reliabilität in der TZ (PZ: κ = 0,81; TZ: κ = 0,80).

Schlussfolgerung PI-RADS v2.1 verbessert die Interrater-Reliabilität in der PZ. Neue Differenzen in der Interrater-Reliabilität bei PI-RADS v2.1 entsprangen hauptsächlich den Läsionen in der TZ, die mit einem Score von 1 oder 2 bewertet wurden. Durch die Kombination der klinisch gleichwertigen PI-RADS-Bewertungen 1 und 2 wird die Interrater-Reliabilität sowohl in der TZ als auch PZ verbessert, sodass eine verbesserte Definition für diese Bewertungen angebracht sein könnte.

Kernaussagen:

  • PI-RADS v2.1 verbessert Interrater-Reliabilität in der PZ, aber nicht in der TZ.

  • Neue Unterschiede entspringen den PI-RADS v2.1-Bewertungen 1 und 2 in der TZ.

  • Die Kombination der PI-RADS v2.1-Bewertungen 1 und 2 verbessert die Interrater-Reliabilität.

  • PI-RADS v2.1 ermöglicht offenbar eine präzisere Deskription von Läsionen in der PZ.

  • Verbesserung der Interrater-Reliabilität in der PZ betont die Bedeutung von geeigneten Deskriptoren.



Publication History

Received: 24 November 2021

Accepted: 29 December 2021

Article published online:
11 May 2022

© 2022. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 
  • References

  • 1 Rosenkrantz AB, Verma S, Choyke P. et al. Prostate Magnetic Resonance Imaging and Magnetic Resonance Imaging Targeted Biopsy in Patients with a Prior Negative Biopsy: A Consensus Statement by AUA and SAR. The Journal of urology 2016; 196: 1613-1618 DOI: 10.1016/j.juro.2016.06.079.
  • 2 Team NGU. National Institute for Health and Care Excellence: Clinical Guidelines. In: Prostate cancer: diagnosis and management. London: National Institute for Health and Care Excellence (UK). Copyright (c) NICE 2019.; 2019:
  • 3 Mottet N, van den Bergh RCN, Briers E. et al. EAU-EANM-ESTRO-ESUR-SIOG Guidelines on Prostate Cancer-2020 Update. Part 1: Screening, Diagnosis, and Local Treatment with Curative Intent. Eur Urol 2020; DOI: 10.1016/j.eururo.2020.09.042.
  • 4 Ahmed HU, El-Shater Bosaily A, Brown LC. et al. Diagnostic accuracy of multi-parametric MRI and TRUS biopsy in prostate cancer (PROMIS): a paired validating confirmatory study. Lancet (London, England) 2017; 389: 815-822 DOI: 10.1016/s0140-6736(16)32401-1.
  • 5 Kasivisvanathan V, Rannikko AS, Borghi M. et al. MRI-Targeted or Standard Biopsy for Prostate-Cancer Diagnosis. The New England journal of medicine 2018; 378: 1767-1777 DOI: 10.1056/NEJMoa1801993.
  • 6 Volkin D, Turkbey B, Hoang AN. et al. Multiparametric magnetic resonance imaging (MRI) and subsequent MRI/ultrasonography fusion-guided biopsy increase the detection of anteriorly located prostate cancers. BJU international 2014; 114: E43-E49 DOI: 10.1111/bju.12670.
  • 7 Komai Y, Numao N, Yoshida S. et al. High diagnostic ability of multiparametric magnetic resonance imaging to detect anterior prostate cancer missed by transrectal 12-core biopsy. The Journal of urology 2013; 190: 867-873 DOI: 10.1016/j.juro.2013.03.078.
  • 8 Padhani AR, Weinreb J, Rosenkrantz AB. et al. Prostate Imaging-Reporting and Data System Steering Committee: PI-RADS v2 Status Update and Future Directions. Eur Urol 2018; DOI: 10.1016/j.eururo.2018.05.035.
  • 9 Le JD, Stephenson S, Brugger M. et al. Magnetic resonance imaging-ultrasound fusion biopsy for prediction of final prostate pathology. The Journal of urology 2014; 192: 1367-1373 DOI: 10.1016/j.juro.2014.04.094.
  • 10 Barentsz JO, Richenberg J, Clements R. et al. ESUR prostate MR guidelines 2012. European radiology 2012; 22: 746-757 DOI: 10.1007/s00330-011-2377-y.
  • 11 Vache T, Bratan F, Mege-Lechevallier F. et al. Characterization of prostate lesions as benign or malignant at multiparametric MR imaging: comparison of three scoring systems in patients treated with radical prostatectomy. Radiology 2014; 272: 446-455 DOI: 10.1148/radiol.14131584.
  • 12 Vargas HA, Hotker AM, Goldman DA. et al. Updated prostate imaging reporting and data system (PIRADS v2) recommendations for the detection of clinically significant prostate cancer using multiparametric MRI: critical evaluation using whole-mount pathology as standard of reference. European radiology 2016; 26: 1606-1612 DOI: 10.1007/s00330-015-4015-6.
  • 13 Greer MD, Shih JH, Lay N. et al. Validation of the Dominant Sequence Paradigm and Role of Dynamic Contrast-enhanced Imaging in PI-RADS Version 2. Radiology 2017; 285: 859-869 DOI: 10.1148/radiol.2017161316.
  • 14 Padhani AR, Weinreb J, Rosenkrantz AB. et al. Prostate Imaging-Reporting and Data System Steering Committee: PI-RADS v2 Status Update and Future Directions. Eur Urol 2019; 75: 385-396 DOI: 10.1016/j.eururo.2018.05.035.
  • 15 Weinreb JC, Barentsz JO, Choyke PL. et al. PI-RADS Prostate Imaging – Reporting and Data System: 2015, Version 2. Eur Urol 2016; 69: 16-40 DOI: 10.1016/j.eururo.2015.08.052.
  • 16 Turkbey B, Rosenkrantz AB, Haider MA. et al. Prostate Imaging Reporting and Data System Version 2.1: 2019 Update of Prostate Imaging Reporting and Data System Version 2. Eur Urol 2019; 76: 340-351 DOI: 10.1016/j.eururo.2019.02.033.
  • 17 Baur AD, Maxeiner A, Franiel T. et al. Evaluation of the prostate imaging reporting and data system for the detection of prostate cancer by the results of targeted biopsy of the prostate. Invest Radiol 2014; 49: 411-420 DOI: 10.1097/rli.0000000000000030.
  • 18 Meng X, Rosenkrantz AB, Mendhiratta N. et al. Relationship Between Prebiopsy Multiparametric Magnetic Resonance Imaging (MRI), Biopsy Indication, and MRI-ultrasound Fusion-targeted Prostate Biopsy Outcomes. Eur Urol 2016; 69: 512-517 DOI: 10.1016/j.eururo.2015.06.005.
  • 19 Epstein JI, Egevad L, Amin MB. et al. The 2014 International Society of Urological Pathology (ISUP) Consensus Conference on Gleason Grading of Prostatic Carcinoma: Definition of Grading Patterns and Proposal for a New Grading System. Am J Surg Pathol 2016; 40: 244-252 DOI: 10.1097/pas.0000000000000530.
  • 20 Donin NM, Laze J, Zhou M. et al. Gleason 6 prostate tumors diagnosed in the PSA era do not demonstrate the capacity for metastatic spread at the time of radical prostatectomy. Urology 2013; 82: 148-152 DOI: 10.1016/j.urology.2013.03.054.
  • 21 Nickel JC, Speakman M. Should we really consider Gleason 6 prostate cancer?. BJU international 2012; 109: 645-646 DOI: 10.1111/j.1464-410X.2011.10854.x.
  • 22 de Rooij M, Israël B, Tummers M. et al. ESUR/ESUI consensus statements on multi-parametric MRI for the detection of clinically significant prostate cancer: quality requirements for image acquisition, interpretation and radiologists' training. European radiology 2020; 30: 5404-5416 DOI: 10.1007/s00330-020-06929-z.
  • 23 Konietschke F, Brunner E. , Hrsg. Nonparametric analysis of clustered data in diagnostic trials: Estimation problems in small sample sizes. 2009
  • 24 Rudolph MM, Baur ADJ, Haas M. et al. Validation of the PI-RADS language: predictive values of PI-RADS lexicon descriptors for detection of prostate cancer. European radiology 2020; 30: 4262-4271 DOI: 10.1007/s00330-020-06773-1.
  • 25 Rosenkrantz AB, Babb JS, Taneja SS. et al. Proposed Adjustments to PI-RADS Version 2 Decision Rules: Impact on Prostate Cancer Detection. Radiology 2017; 283: 119-129 DOI: 10.1148/radiol.2016161124.
  • 26 Hansen NL, Koo BC, Warren AY. et al. Sub-differentiating equivocal PI-RADS-3 lesions in multiparametric magnetic resonance imaging of the prostate to improve cancer detection. Eur J Radiol 2017; 95: 307-313 DOI: 10.1016/j.ejrad.2017.08.017.
  • 27 Tamada T, Kido A, Takeuchi M. et al. Comparison of PI-RADS version 2 and PI-RADS version 2.1 for the detection of transition zone prostate cancer. Eur J Radiol 2019; 121: 108704 DOI: 10.1016/j.ejrad.2019.108704.
  • 28 Brembilla G, DellʼOglio P, Stabile A. et al. Interreader variability in prostate MRI reporting using Prostate Imaging Reporting and Data System version 2.1. European radiology 2020; 30: 3383-3392 DOI: 10.1007/s00330-019-06654-2.
  • 29 Bhayana R, OʼShea A, Anderson MA. et al. PI-RADS versions 2 and 2.1: Interobserver Agreement and Diagnostic Performance in Peripheral and Transition Zone Lesions Among Six Radiologists. Am J Roentgenol 2020; DOI: 10.2214/ajr.20.24199.
  • 30 Rudolph MM, Baur ADJ, Cash H. et al. Diagnostic performance of PI-RADS version 2.1 compared to version 2.0 for detection of peripheral and transition zone prostate cancer. Sci Rep 2020; 10: 15982 DOI: 10.1038/s41598-020-72544-z.
  • 31 Polanec S, Helbich TH, Bickel H. et al. Head-to-head comparison of PI-RADS v2 and PI-RADS v1. Eur J Radiol 2016; 85: 1125-1131 DOI: 10.1016/j.ejrad.2016.03.025.
  • 32 Schaudinn A, Gawlitza J, Mucha S. et al. Comparison of PI-RADS v1 and v2 for multiparametric MRI detection of prostate cancer with whole-mount histological workup as reference standard. Eur J Radiol 2019; 116: 180-185 DOI: 10.1016/j.ejrad.2019.04.012.