Keywords critical test results - pathology information systems - hospital communication systems
- automated electronic alert notification - clinically significant test results
Background and Significance
Background and Significance
Failure to follow-up test results has well-documented consequences, including delayed
diagnosis and treatment, malpractice litigation, added health care costs, and potential
patient harm.[1 ]
[2 ]
[3 ]
[4 ]
[5 ] In ambulatory settings, examples of test results that typically “fall through the
cracks” include nonurgent, clinically significant results of radiology studies (e.g.,
incidental pulmonary nodules), pathology specimens, microbiology cultures, chemistry
and hematology tests, and cardiology studies (e.g., echocardiograms).
Effectively communicating clinically significant test results (CSTRs) is a Joint Commission's
National Patient Safety Goal.[6 ] CSTRs have been defined as “any result that requires further clinical action to
avoid morbidity or mortality, regardless of the urgency of that action.”[7 ] Closed-loop communication is a key process to ensure that CSTRs are acknowledged
and acted upon appropriately. Breakdowns in the closed-loop communication process
are often due to failure to acknowledge and follow-up test results.[8 ]
[9 ]
[10 ]
[11 ]
[12 ]
[13 ]
[14 ] Automated systems that alert providers have been shown to improve communication
and time to appropriate management of CSTRs.[15 ]
We previously implemented the Alert Notification of Critical Results (ANCR) system
to facilitate closed-loop communication of radiology CSTRs. Alerts are transmitted
through either synchronous (i.e., pager) or asynchronous (i.e., secure e-mail) mechanisms,
both of which require acknowledgment to close the communication loop.[16 ]
[17 ] This Web-based software has been optimized for mobile access and allows for a one-click
acknowledgment process that leverages our secure institutional single sign-on process.
Asynchronous, noninterruptive communication is particularly helpful for clinically
significant but nonurgent findings (e.g., pulmonary nodules). Implementing ANCR has
been shown to improve communication of critical radiology results and enhance adherence
to institutional policies regarding CSTRs, with 95% of reports with critical results
adherent to policy and sustained improvement over the 4 years after implementation.[17 ]
[18 ] ANCR was widely adopted by radiologists—it was used for 81% of all communicated
reports by 18 months after implementation; usage persisted for the remainder of the
study and afterwards.[17 ]
With the successful adoption and associated improvement in CSTR management for radiology,[16 ]
[17 ]
[19 ] there was much interest in expanding the ANCR to other results-generating areas
(RGAs) at our institution that struggled with closed-loop communication of CSTRs (e.g.,
RGAs that have had malpractice claims associated with failure to follow-up high-risk
results such as new malignancies), and whose complexity require physician assessment
of the clinical situation to determine if a result is clinically significant. For
example, pathology results, such as reports of biopsy specimens that demonstrate new
malignancies, may not be immediately life-threatening but do require subsequent and
timely action (e.g., follow-up with a specialist, surgical intervention, or chemotherapy)
to prevent morbidity.
Objective
The aims of this study were to: (1) implement ANCR in a pathology RGA, and (2) evaluate
the impact of ANCR on closed-loop communication of a specific type of CSTRs, i.e.,
new malignancies. We hypothesized that ANCR, if properly integrated into the workflow,
would be used to facilitate more timely, closed-loop communication of pathology CSTRs.
Materials and Methods
Study Site and Cohort
This Institutional Review Board-approved study was performed at a 150-bed community
hospital affiliated with an academic medical center in a department of 4 pathologists
performing over 14,500 cytopathology and surgical pathology exams annually. A total
of 30,774 cytopathology or surgical pathology specimens were submitted for 23,948
adult patients during a 13-month preintervention and 13-month postintervention period
from January 2012 through April 2014 ([Fig. 1 ]).
Fig. 1 Study cohort. NLP, natural language processing.
As part of an ongoing institution-wide initiative to standardize test results across
our health care network, CSTRs were classified as Level 1, 2, or 3 based on predetermined
criteria ([Table 1 ]).[7 ]
[20 ] Pathologists could use these criteria to determine an appropriate level of urgency
at the time of specimen interpretation. Because the majority of “Level 3” pathology
results were new malignancies, we focused our evaluation on new malignancies. Malignancies
were considered new if not previously sampled. For example, a malignancy in a biopsy
of a breast lesion would be considered new whereas malignancy in the subsequent lumpectomy
or mastectomy specimen would not be considered new. We excluded “Level 1” and “Level
2” CSTRs as these were exceedingly rare in our pathology RGA.
Table 1
CSTR criteria and acknowledgment time frames
Level 1 : Critical, for example, fat in endometrial curettage or endoscopic polypectomy; acknowledgment
of alert required by referring/ordering provider within 1 hour; reminders sent hourly
until acknowledged
Level 2 : Urgent, for example, leukocytoclastic vasculitis; acknowledgment of alert required
by referring/ordering provider within 24 hours; reminders sent daily until acknowledged
Level 3 : Nonurgent but clinically significant, for example, new malignancy; acknowledgment
of alert required by referring/ordering provider within 15 days; reminders sent weekly
until due then daily until acknowledged
Abbreviation: CSTR, clinically significant test result.
Study cohort selection is summarized in [Fig. 1 ]. To identify pathology reports with malignancies, we developed a natural language
processing (NLP) algorithm using a publicly available document retrieval toolkit,
Information from Searching Content with an Ontology-Utilizing Toolkit (iSCOUT).[21 ] We previously used iSCOUT to retrieve radiology reports containing critical results.[22 ]
[23 ]
[24 ] For this study, we used iSCOUT to identify reports with malignancies and exclude
reports that negated the concept, using phrases such as “no carcinoma” or “carcinoma
was not seen.” See Appendix A for a complete list of terms. We validated the NLP algorithm by manually reviewing
1,000 randomly selected reports to determine the presence of malignancy: 169 true-positive,
15 false-positive, 816 true-negative, and zero false-negative pathology reports documenting
malignancies were identified, yielding a sensitivity of 100%, specificity of 98%,
and precision of 92%.
Usual Care
Prior to intervention, to facilitate closed-loop communication, pathologists would
open a separate application either to page or e-mail the referring provider with critical
results, and then document acknowledgment in the pathology report once the referring
provider responded. However, they would still rely on manual reminders to follow-up
with those referring providers who did not respond and acknowledge the CSTR. Furthermore,
if acknowledgment was delayed, the pathologist would need to addend the pathology
report to document the provider's response. This manual workflow likely contributed
to missed CSTRs at this pathology RGA.
Intervention
ANCR is a public domain, Web-based application developed to facilitate closed-loop
communication of critical radiology test results (including nonurgent, CSTRs) and
has been previously described.[16 ] We adapted ANCR for use within a single pathology department. In a series of meetings,
we engaged administrative leadership and all practicing pathologists to understand
their clinical workflow, identify potential access points, review policies, and set
expectations for use according to our institutional policy.
The new system allowed pathologists to launch ANCR in the context of the patient's
specimen being reported ([Fig. 2 ]) by clicking a button within their current laboratory information system (CoPath,
Cerner, North Kansas City, Missouri, United States). The pathologist identified a
CSTR according to the institutional policy (see [Table 1 ]), and alerted the referring provider via pager for Level 1 and Level 2 results,
and via pager or secure e-mail for Level 3 results. The phrase “This report was entered
into the Alert Notification of Critical Results (ANCR) System at the time of signout”
was automatically documented in the pathology report, indicating use of the system
(see [Appendix B ]). The referring provider could then call the pathologist or follow a link in a secure
e-mail directing him/her to the ANCR application to acknowledge the CSTR. Importantly,
using ANCR, the pathologist did not have to rely on a manual process for remembering
to follow-up with referring providers who did not acknowledge a critical result—the
system sent automatic reminders (e.g., weekly until due then daily for Level 3 alerts,
see [Table 1 ]) to both the pathologist and referring provider (e.g., surgeon, oncologist, gynecologist
who ordered or performed the biopsy) until the alert was acknowledged. Furthermore,
a clinical administrator for pathology monitored a worklist of unacknowledged alerts
to ensure all alerts were closed.
Fig. 2 Laboratory information systems integration. The highlighted button launches Alert
Notification of Critical Results (ANCR) in context of the patient and exam being reported.
Main Outcomes and Data Collection
The primary outcome was documented acknowledgment within 15 days for new malignancies
(adherence to our institutional policy for Level 3 pathology CSTRs).[7 ] Because no preintervention and few postintervention results were acknowledged in
ANCR, chart review was used as the gold standard to identify evidence of documented
acknowledgment. Any documentation in clinical notes, result review notes, telephone
encounters, pathology reports, and/or procedure notes that identified or addressed
the key finding was considered acknowledgment (i.e., stating the key finding, notifying,
and/or communicating with the patient or another health care provider, establishing
or changing a diagnosis, recommending, ordering, or cancelling a test or study, and/or
referring the patient to another provider). The secondary outcome was time to acknowledgment
(measured in days), defined as the interval between submission of the specimen and
documentation by a health care provider.
A random sample of 400 pathology reports from pre- and postintervention periods that
were NLP positive for malignancy was manually reviewed. The electronic health record
(EHR) was reviewed to determine if the malignancies were new and identify when new
malignancies were first acknowledged. The first 10% were independently reviewed by
two physician investigators and agreement in documented alert acknowledgment was calculated
using the kappa statistic. All discrepancies were resolved; the final criteria were
used by a single physician investigator to review the remaining cases.
We also assessed user reports of ANCR adoption. Pathologists were surveyed about use
of ANCR 4 months after implementation. The survey ([Appendix B ]) addressed usage patterns, including the types of findings and perceived proportion
of reports that triggered use, as well as attitudes toward using ANCR. Responses were
categorical, with the option of free-text response. We assessed satisfaction by asking
whether ANCR had a positive impact on workflow. Finally, to quantify use of ANCR according
to the institutional policy, we measured the proportion of new and nonnew malignancies
in our sample for which an ANCR alert was sent.
Power Sample and Statistical Analysis
Based on prior studies, we expected to observe electronic documentation of acknowledgment
in 40% of patients with actionable test results when a provider was aware of the result.[13 ]
[25 ]
[26 ]
[27 ] We previously showed that e-mail notification of actionable test results doubles
awareness at 72 hours after becoming available.[25 ]
[26 ] Therefore, we anticipated that ANCR would result in an increase of documented acknowledgment
from 20 to 40%. To detect this change with 80% power and an α of 0.05, we estimated that we would need to review 180 actionable pathology results
(90 pre- and 90 postintervention). We reviewed 50 randomly sampled reports that were
NLP-positive for malignancy of which 30 (60%) were considered new (and thus CSTRs).
Therefore, we would need to review a total of 300 randomly sampled reports that are
NLP-positive for malignancy to identify those with new malignancies. To be conservative,
we sampled 400 NLP-positive reports: 200 pre- and 200 postintervention.
The proportion of reports with documented acknowledgment was compared in the pre-
and postintervention periods using Fisher's exact test. Wilcoxon rank sum was used
to compare times to acknowledgment for submitted specimens in the pre- and postintervention
periods as well as between ANCR-acknowledged and EHR-acknowledged alerts in the postintervention
period. The proportion of postintervention reports of NLP positive for malignancy
for which an ANCR was sent was compared among those with new versus previously identified
malignancy using Fisher's exact test. A two-sided p -value of < 0.05 was used to determine statistical significance.
Results
Study Cohort and Sample
Of 30,774 pathology reports (23,948 patients), 5,595 (18%; 4,099 patients) were NLP
positive for malignancy ([Fig. 1 ]). Patient characteristics for the study cohort and sample are presented in [Table 2 ]. In general, patient characteristics in the study cohort and sample were similar
in the pre- and postintervention periods. ANCR alerts were generated on a similar
proportion of postintervention patients in the study and sample cohort. Of the 2,802
postintervention reports that were NLP positive for malignancy, ANCR alerts were sent
on 213 (7.6%). Of the 200 randomly sampled NLP malignancy-positive postintervention
reports, new malignancies were identified in 103 (51.5%) cases based on chart review;
of these 103, ANCR alerts were generated on 15 (14.6%), a greater proportion than
for all 200 NLP-positive reports (7.6%, p = 0.010). Although pathologists had the option of sending Level 3 ANCR alerts by
pager or secure e-mail, all alerts were sent via e-mail.
Table 2
Demographics of patients' pathology reports
NLP+ for malignancy (Pre) n = 2,793
NLP+ for malignancy (Post) n = 2,802
Preintervention sample, n = 200
Postintervention sample, n = 200
No. of unique patients
2,124
2,126
196
195
Age (y)
61 ± 14
62 ± 14
61 ± 14
61 ± 15
Female
75% (2,093)
72% (2,004)
76% (152)
75% (149)
Race
White
81% (2,264)
80% (2,238)
80% (160)
79% (158)
Black
5.6% (156)
5.3% (148)
3.0% (6)
7.5% (15)
Hispanic
2.5% (71)
2.7% (77)
5.5% (11)
3.5% (7)
Asian
1.9% (53)
1.5% (41)
3.0% (6)
1.0% (2)
Other
8.9% (249)
11% (298)
8.5% (17)
9.0% (18)
Insurance
Government
36% (1,004)
37% (1,036)
31% (62)
36% (71)
Private
64% (1,774)
63% (1,756)
69% (137)
64% (128)
Other
0.5% (15)
0.4% (10)
0.5% (1)
0.5% (1)
# of reports with new malignancies validated by chart review
107 (3.8%)
103 (3.7%)
107 (54%)
103 (52%)
# of reports on which an ANCR alert was generated
N/A
213 (7.6%)
N/A
15 (7.5%)[a ]
Abbreviations: ANCR, Alert Notification of Critical Results; NLP, natural language
processing.
Note: Multiple specimens resulting in one pathology report were treated as single
specimens.
a In the postintervention sample, one additional ANCR alert was generated on a false-positive
NLP+ pathology report (i.e., without a new malignancy) based on independent chart review;
hence, there were 16 (8.0%) NLP+ pathology reports on which an ANCR alert was generated in the postintervention sample.
Rate of Documented Acknowledgment
Rates of acknowledgment of pathology reports with new malignancies as documented in
the EHR were similar in the pre- and postintervention periods ([Table 3 ]). For a random subsample of 40 reports, reviewers agreed that a new malignancy was
present in 35 (Kappa = 0.73, 95% confidence interval, 0.51–0.94).
Table 3
Estimates of reports with documented acknowledgment
Preintervention (n = 200)
Postintervention (n = 200)
p -Value
New malignancies with acknowledgment (% [95% CI])
106/107
(99 [95–100]%)
100/103
(97 [92–99]%)
0.3620
New malignancies with acknowledgment within 15 d (% [95% CI])
98/107
(92 [86–97]%)
89/103
(86 [80–93]%)
0.2294
Abbreviation: CI, confidence interval.
Time to Documented Acknowledgment
The median time to acknowledgment for all new malignancies was 7 days (interquartile
range [IQR], 3, 11) preintervention and 6 days (IQR, 2, 10) postintervention (p = 0.5083). In the postintervention period, when ANCR was used to communicate and
acknowledgment was documented in the EHR, median time to acknowledgment was significantly
shorter (2 vs. 7 days, p = 0.0351) than when an ANCR alert was not generated ([Table 4 ]).
Table 4
Postintervention: differential impact of ANCR on time to acknowledgment
ANCR alert (n = 15)
No ANCR alert (n = 85)
p -Value
New malignancies, days until acknowledgment (median, [IQR])
2 [1, 6]
7 [2.5, 10.5]
0.0351
Abbreviations: ANCR, Alert Notification of Critical Results; IQR, interquartile range.
In the postintervention sample, ANCR alerts were more often generated on reports with
new malignancies compared with reports with malignancies that were not new (14.6%
vs. 1.0%, p < 0.01); however, ANCR alerts were not sent on a large proportion of reports with
new malignancies (85.4%, n = 88) that should have been designated as clinically significant according to our
institutional criteria.
ANCR Pathology Adoption and Use
The four pathologists who comprised the pathology department during the study period
(50% female, 6–27 years of experience) were surveyed and all completed the questionnaire.
All generated ANCR alerts on less than 25% of reports. Findings that prompted ANCR
use included malignancies (4 pathologists), corrected or addendum reports (3 pathologists),
and discrepancies between findings and diagnoses documented in preliminary and finalized
reports (3 pathologists). All pathologists felt that the ANCR system positively impacted
their workflow: of the three pathologists who provided free-text clarifications, all
cited increased efficiency in communicating CSTRs and one specifically stated that
ANCR was more efficient than paging or e-mail.
Discussion
We observed that electronic documentation of acknowledgment for a specific type of
Level 3 CSTR was sustained at a high rate for a pathology RGA pre- and post-ANCR implementation.
We observed a significant decrease in time to documented acknowledgment associated
with use of ANCR. Although pathologists uniformly had a favorable impression of ANCR
with regard to improving workflow for communication of CSTRs, they used ANCR to facilitate
closed-loop communication for only a small proportion of CSTRs in our cohort, defined
as Level 3 according to the institutional criteria.
The rate of documented acknowledgment within 15 days was unexpectedly high. Prior
studies reported rates between 3 and 50%.[13 ]
[25 ]
[26 ]
[27 ] There are several explanations for this finding: (1) users may have a lower threshold
for documenting communication about new malignancies on pathology specimens as compared
with other types of results (e.g., microbiology, laboratory tests); (2) new malignancies
tend to require complex follow-up actions which are typically documented in the EHR
(e.g., consults, additional surgeries, chemotherapy, radiation); (3) nearly half of
the malignancies in our cohort were breast cancer, a condition with a well-established
workflow for follow-up at this institution; and (4) medicolegal concerns.
We did not observe an improvement in our primary outcome after implementation of ANCR.
This may be explained partly by ceiling effects; however, lack of improvement could
also be explained by a low rate (∼15%) of ANCR use to facilitate closed-loop communication
for the new malignancies sampled in our cohort. Although we held preintervention meetings
to review workflow and policy, we did not conduct a formal assessment to understand
how ANCR would be used and identify potential usage barriers. Also, in contrast to
our other deployments, we did not implement performance reporting to encourage pathologists
to use ANCR to facilitate closed-loop communication of new malignancies. Routine performance
reporting is an important feedback mechanism that was associated with improved rates
of adherence to institutional policy in our prior efforts.[18 ]
The shorter time to acknowledge CSTRs communicated via ANCR compared with those communicated
via routine pathology reports can be attributed to ANCR's Web-based and mobile workflow,
and one-click acknowledgment process. Because responsible providers are actively notified
of CSTRs via a “push” (e.g., e-mail notification external to the EHR), they are not
required to open the EHR to search for new results. Active notification is important
for critical results because earlier acknowledgment should lead to more timely action,
especially when required follow-up is more urgent.[28 ]
[29 ] When follow-up action is not urgent, as is often the case for Level 3 CSTRs, acknowledgment
may not be documented until the action is actually performed (which may be days after
the result is finalized). Thus, in theory, the decreased time to acknowledgment that
we observed for ANCR may improve time to action for this subset of test results.
Overall, pathologists had a favorable impression of ANCR: strong support from administrative
leadership, clear expectations for use through institutional policy, and tight integration
into clinical workflow were key factors identified from our qualitative assessment.
Nonetheless, despite uniform adoption of ANCR by all pathologists in our study, ANCR
was not uniformly used to facilitate closed-loop communication of CSTRs in accordance
with institutional policy. The etiology of the discrepancy was likely multifactorial,
most probably due to differences in interpretation of policies at the local and enterprise
level regarding types of results considered critical. Specifically, per local departmental
policy, only an unexpected malignancy should trigger an alert, whereas per enterprise policy, any new malignancy should trigger an alert. Thus, a departmental pathologist may decide to
send an alert just for the subset of new malignancies that were truly not anticipated
(i.e., clinical suspicion for cancer was not the indication for biopsy). Alternatively,
the pathologist may not remember the policy or decide to notify in another way (manual
call/page) without using ANCR. In addition to implementing performance reporting as
described above, additional educational training, further harmonization of policies
at the local RGA and enterprise levels, and mechanisms to ensure accountability regarding
best practices for using closed-loop communication technologies are often necessary
to encourage adoption.
Test result management in EHRs remains a source of dissatisfaction[30 ] and poses a significant safety concern.[31 ] Further, many providers create potentially unsafe workarounds to fill gaps in functionality.[32 ] ANCR minimizes this by providing a modern and more familiar Web-based and mobile
user experience to those individuals receiving alerts. Indeed, referring providers
are much more likely to acknowledge radiology result alerts via ANCR than the EHR
when alerted for the same result using both mechanisms and when acknowledgment via
one mechanism closed the alert in both systems.[20 ] There is no reason to suspect that referring providers would find ANCR alerts unacceptable
for pathology test results as we have described. Furthermore, ANCR worklists serve
as an important memory aid: pathologists can track all alerts, including attempts
at escalating those that remain unacknowledged, and referring providers can track
both unacknowledged and acknowledged alerts that require follow-up. Also, ANCR improves
efficiency of identifying critical pathology reports for referring providers by automated
flagging through integration with the EHR. Finally, by ensuring receipt of alerts,
ANCR fulfills a Joint Commission requirement for closed-loop communication of critical
results.[6 ]
Our study has several limitations. First, because it was performed at a single institution,
our findings may not be generalizable to other institutions with different workflows
and policies. Second, although our sampling strategy was adequate, the low rate of
ANCR alerts generated for new malignancies may have hindered our ability to detect
an impact on our primary outcome. Third, we did not evaluate use of ANCR by referring
providers to acknowledge critical findings. Fourth, in our sample size determination,
we underestimated the rate of documentation of acknowledgment for new malignancies
and unexpectedly encountered ceiling effects during our study. Nonetheless, we still
were able to observe improvement in time to documented acknowledgment. Fifth, this
study focused on evaluating the effect of a specific technology on closed-loop communication
pre- and postimplementation—we did not measure other factors that could have impacted
acknowledgment of new malignancies. Finally, our definition of acknowledgment included
acting on a critical result; thus, it was not possible to analyze acknowledgment and
action separately.
Conclusion
In summary, we observed improvement in time to documented acknowledgment after extending
ANCR to facilitate closed-loop communication of nonurgent pathology CSTRs. We believe
that routine use of performance reporting and educational/feedback initiatives during
implementation could have encouraged more consistent use of ANCR according to institutional
CSTR policy. Nonetheless, understanding how closed-loop communication technologies
such as ANCR are used by health care providers for different result types and in different
health care settings will help shape future interventions to improve follow-up of
CSTRs, while addressing the heterogeneity in test result follow-up practices. Future
studies should focus on evaluating impact on other types of pathology CSTRs, and expand
use of ANCR or similar tools to other RGAs (e.g., echocardiology) and care settings
(e.g., care transitions). Importantly, formal usability assessments should be conducted
to thoroughly understand implementation barriers and facilitators. Clarifying institutional
policy, harmonizing expectations among RGAs, and improving communication is paramount
to ensure successful implementation of technologies to facilitate closed-loop communication
of test results.[33 ]
Clinical Relevance Statement
Clinical Relevance Statement
A Web-based software tool developed to facilitate communication of radiology CSTRs
can be successfully expanded to pathology, garnering a favorable impression from pathologists,
reducing the time to CSTR acknowledgment, and setting the stage for expansion to other
areas as an enterprise-wide solution. Despite a positive reception, the software was
only used for a minority of new malignancies, all of which would be considered CSTRs
by enterprise policy. This highlights the need for CSTR communication policy harmonization
between local results generating areas and the enterprise, establishment of performance
reviews, creation of education initiatives, and understanding the heterogeneity of
test result follow-up practices.
Multiple Choice Questions
Multiple Choice Questions
Which of the following is true regarding the Joint Commission's National Patient and
Safety Goal for communication of critical results (NPSG.02.03.01)?
It leaves the definition of critical results up to local organizational leadership.
It specifies the acceptable length of time between the availability and reporting
of critical results.
It applies to tests and diagnostic procedures in the inpatient and emergency care
settings, but not the outpatient setting.
It provides a list of acceptable provider roles which can receive and acknowledge
critical results.
Correct Answer: The correct answer is option a. The Joint Commission's National Patient and Safety
Goal for communication of critical results includes collaboration with organization
leaders to define critical results, determine who gives and receives critical results
communication, and set the time by which communication must occur. It applies in all
patient care settings.
A Web-based application was developed to facilitate closed-loop communication of clinically
significant test results was associated with:
High rates of use of the system by pathologists to report new malignancies.
Improved rates of acknowledgment by ambulatory clinicians of pathology reports with
new malignancies.
No change in reported workflow efficiency by pathologists.
A reduction in median time to acknowledgment of a pathology report of new malignancy.
Correct Answer : The correct answer is option d. When the system was used, there was a significant
reduction in median time to acknowledgment and all pathologists thought the system
improved their efficiency. However, the system was not highly used and there was no
significant change in the high rates of acknowledgment before and after implementation.
Appendix A: iSCOUT Search terms
adenocarcinoma(s), atypical ductal hyperplasia, atypical lobular hyperplasia, barrett,
barrett's, barretts, carcinoid(s), carcinoma(s), dcis, high grade, high-grade, gastrointestinal
stromal tumor(s), gist, leiomyosarcoma(s), leukemia(s), lymphoma(s), malignancies,
malignancy, malignant, melanoma(s), metastases, metastasis, metastatic, neoplasm(s),
plasmacytoma(s), sarcoma(s), tumor(s).
Appendix B: ANCR Pathology Survey and Focus Group
Approximately how many pathology specimens do you review on a weekly basis? __________________
Of all the pathology samples you review on a weekly basis, on approximately what %
do you generate an ANCR alert?
< 25%
26–50%
51–75%
> 76%
In the inpatient setting, do you send ANCR alerts to:
Responding clinician (e.g., intern).
Responsible clinician (attending).
Another provider? __________________.
In retrospect, have you ever forgotten to generate an ANCR alert when once should
have been generated?
YES
NO
Which types of findings do you typically generate alerts on?
Precancerous specimens.
Malignancies.
Corrected or addendum reports.
Discrepant diagnosis (e.g., intra-op vs. final, rapid vs. final, intra vs. extra departmental
review, etc.). Please specify:_____________________________________________________________.
Other(s):_______________________________________________________________________.
Has ANCR impacted your workflow POSITIVELY or NEGATIVELY (circle one)? Please describe
how:
__________________
On approximately what percentage of your generated alerts does the designated receiver
acknowledge the ANCR alert within the appropriate timeframe?
< 25%
26–50%
51–75%
> 76%
On approximately what percentage of your generated alerts do you have to close the
ANCR alert yourself?
< 25%
26–50%
51–75%
> 76%
Briefly describe or list any barriers you have experienced to using the ANCR tool?
List any recommendations you have for other results generating areas with regard to
implementing and using ANCR to facilitate closed-loop communication for test results?
Comments, suggestions, and feedback.
