CC BY-NC-ND 4.0 · Appl Clin Inform 2018; 09(01): 062-071
DOI: 10.1055/s-0037-1617451
Research Article
Schattauer GmbH Stuttgart

Improving the Accuracy of a Clinical Decision Support System for Cervical Cancer Screening and Surveillance

K.E. Ravikumar
Kathy L. MacLaughlin
Marianne R. Scheitel
Maya Kessler
Kavishwar B. Wagholikar
Hongfang Liu
Rajeev Chaudhry
Further Information

Publication History

12 July 2017

25 November 2017

Publication Date:
24 January 2018 (online)


Background Clinical decision support systems (CDSS) for cervical cancer prevention are generally limited to identifying patients who are overdue for their next routine/next screening, and they do not provide recommendations for follow-up of abnormal results. We previously developed a CDSS to automatically provide follow-up recommendations based on the American Society of Colposcopy and Cervical Pathology (ASCCP) guidelines for women with both previously normal and abnormal test results leveraging information available in the electronic medical record (EMR).

Objective Enhance the CDSS by improving its accuracy and incorporating changes to reflect the latest revision of the guidelines.

Methods After making enhancements to the CDSS, we evaluated the performance of the clinical recommendations on 393 patients selected through stratified sampling from a set of 3,704 patients in a nonclinical setting. We performed chart review of individual patient's record to evaluate the performance of the system. An expert clinician assisted by a resident manually reviewed the recommendation made by the system and verified whether the recommendations were as per the ASCCP guidelines.

Results The recommendation accuracy of the enhanced CDSS improved to 93%, which is a substantial improvement over the 84% reported previously. A detailed analysis of errors is presented in this article. We fixed the errors identified in this evaluation that were amenable to correction to further improve the accuracy of the system. The source code of the updated CDSS is available at

Conclusion We made substantial enhancements to our earlier prototype CDSS with the updated ASCCP guidelines and performed a thorough evaluation in a nonclinical setting to improve the accuracy of the CDSS. The CDSS will be further refined as it is utilized in the practice.

Protection of Human and Animal Subjects

This study does not involve any experiments involving human and animal subjects. The institutional review board at the Mayo Clinic, Rochester, MN, approved this study.


We acknowledge the funding from the Agency for Healthcare Research and Quality (AHRQ), grant number R21H S022911–01, NLP enabled decision support for cervical cancer screening and surveillance that supported this work.