Abstract
Acute stroke alerts are frequently triggered by conditions unrelated to cerebrovascular
disease, resulting in false positives that burden clinical teams and contribute to
diagnostic ambiguity. At a large academic center, we developed ScanNER v2, a machine
learning (ML) model based on large-language models (LLMs) and structured clinical
data to predict the presence of acute cerebrovascular disease (ACD) in approximately
16,000 stroke alerts occurring over 10 years with an area under the receiver-operating
curve and F1 score of 0.72 and overall positive predictive value of 0.68. In this
perspective, we outline a practical framework for operationalizing this model within
hospital-based stroke systems. We first describe our health-system experience developing
and validating an AI-enabled pipeline, named “ScanNER 2,” then take the point of view
of two implementation angles (high sensitivity and high specificity), outlining the
operational and clinical tradeoffs for each approach. We also highlight challenges
related to implementation, clinical governance, workflow integration, and equity,
emphasizing guardrails required for responsible deployment. As stroke centers increasingly
adopt AI-assisted tools, this type of thought experiment is essential to ensure that
such ML-based innovations effectively enhance the core mission of delivering timely,
high-quality acute stroke care.
Keywords
machine learning - LLM - large language model - implementation - acute stroke - predictive
modeling - diagnostic acceleration
