The Accuracy of Hestia and Simplified PESI to Predict the Prognosis in Pulmonary Embolism: Systematic Review with Meta-analysis

Introduction  Pulmonary embolism (PE) patients at low risk of early complications may be considered for early discharge or home treatment. Last decades evidence has been growing about the safety of several clinical prediction rules for selecting those patients, such as simplified Pulmonary Embolism Severity Index (sPESI) and Hestia Criteria. The aim of this review was to compare the safety of both strategies regarding 30-days mortality, venous thromboembolism recurrence and major bleeding. Methods  A systematic literature search was conducted using MEDLINE, CENTRAL and Web of Science on 6 th January 2022. We searched for studies that applied both Hestia Criteria and sPESI to the same population. Sensitivity, specificity and diagnostic odds ratio were calculated for both stratification rules. Both Hestia and sPESI criteria of low risk were evaluated to set the number of patients that could be misclassified for each 1000 patients with PE. The estimates were reported with their 95% confidence intervals (95%CI). Results  This systematic review included 3 studies. Only mortality data was able to be pooled. Regarding mortality, the sensitivity, specificity and diagnostic odds ratio was 0.923 (95%CI: 0.843–0.964), 0.338 (95%CI: 0.262–0.423) and 6.120 (95%CI: 2.905–12.890) for Hestia Criteria; and 0.972 (95%CI: 0.917–0.991), 0.269 (95%CI: 0.209–0.338) and 12.738 (95%CI: 3.979–40.774) for sPESI score. The negative predictive values were higher than 0.977. The risk of misclassification of high-risk patients in low risk was 5 (95%CI: 3–11) with Hestia and 2 (95%CI: 1–6) with sPESI, for each 1000 patients with PE in terms of mortality. Conclusion  The risk of misclassification of patients presenting with low-risk pulmonary embolism with the intent of early discharge or home treatment with both Hestia Criteria and sPESI score is low and these data supports methods for this purpose.


Topic
No. Item Location where item is reported 13f Describe any sensitivity analyses conducted to assess robustness of the synthesized results.

Reporting bias assessment
14 Describe any methods used to assess risk of bias due to missing results in a synthesis (arising from reporting biases).

N/A
Certainty assessment

15
Describe any methods used to assess certainty (or confidence) in the body of evidence for an outcome.

Study selection
16a Describe the results of the search and selection process, from the number of records identified in the search to the number of studies included in the review, ideally using a flow diagram.

5,7
16b Cite studies that might appear to meet the inclusion criteria, but which were excluded, and explain why they were excluded. 7 Study characteristics 17 Cite each included study and present its characteristics. 7 Risk of bias in studies 18 Present assessments of risk of bias for each included study. 8

Results of individual studies
19 For all outcomes, present, for each study: (a) summary statistics for each group (where appropriate) and (b) an effect estimate and its precision (e.g., confidence/credible interval), ideally using structured tables or plots. 8,9,10,11 Results of syntheses 20a For each synthesis, briefly summarize the characteristics and risk of bias among contributing studies.

7,8
20b Present results of all statistical syntheses conducted. If meta-analysis was done, present for each the summary estimate and its precision (e.g., confidence/credible interval) and measures of statistical heterogeneity. If comparing groups, describe the direction of the effect. 24a Provide registration information for the review, including register name and registration number, or state that the review was not registered.  Patients (setting, intended use of index test, presentation, prior testing): Consecutive patients at least 18 years of age admitted to the hospital with a primary diagnosis of acute pulmonary embolism. There were excluded subjects submitted to thrombolysis and/or pulmonary embolectomy within the first 48h after the acute event.
The index tests aim to identify patients who are at low risk for early mortality, to consider early discharge and provide an anticoagulation strategy in an outpatient setting. All patients were managed according to usual clinical practice for that institution (Hartford Hospital, Connecticut, USA). The index tests were applied a posteriori, using information collected as close as possible to the index pulmonary embolism. For the diagnosis of the pulmonary embolism were used perfusion-ventilation lung scan, computed tomography pulmonary angiography or pulmonary angiography. On the event of the previous testing were non-diagnostic, compression ultrasonography of the ower extremities was performed.
Index test(s): Hestia criteria, simplified Pulmonary Embolism Severity Index (sPESI), Pulmonary Embolism Severity Index, In-hospital Mortality for Pulmonary Embolism using Claims Data (IMPACT) Reference standard and target condition: The reference standard for mortality within the first 30 days of pulmonary embolism presentation is mortality itself.
Phase 2: Draw a flow diagram for the primary study TH Open © 2022. The Author(s).

A. Risk of Bias
Describe methods of patient selection: There were included consecutive patients at least 18 years of age admitted to the hospital with a primary diagnosis of acute pulmonary embolism. There were excluded subjects submitted to thrombolysis and/or pulmonary embolectomy within the first 48h after the acute event, since this put them in a high risk category according to current guidelines. From the 30-day mortality cohort were excluded patients admitted posteriorly to January 2014 on the impossibility of accessing mortality data from the Social Security Death Index platform, due to changes in rules regarding access to data.

B. Concers regarding applicability
Describe included patients (prior testing, presentation, intended use of index test and setting): Consecutive patients at least 18 years of age admitted to the hospital with a primary diagnosis of acute pulmonary embolism. There were excluded subjects submitted to thrombolysis and/or pulmonary embolectomy within the first 48h after the acute event.
The index tests aim to identify patients who are at low risk for early mortality, to consider early discharge and provide an anticoagulation strategy in an outpatient setting. All patients were managed according to usual clinical practice for that institution (Hartford Hospital, Connecticut, USA). The index tests were applied a posteriori, using information collected as close as possible to the index pulmonary embolism. For the diagnosis of the pulmonary embolism were used perfusion-ventilation lung scan, computed tomography pulmonary angiography or pulmonary angiography. On the event of the previous testing were non-diagnostic, compression ultrasonography of the ower extremities was performed.
Is there concern that the included patients do not match the review question? Concern: Low Phase 3: Risk of bias and applicability judgements QUADAS-2 is structured so that 4 key domains are each rated in terms of risk of bias and the concern regarding applicability to the research question (as defined above). Each key domain has a set of signaling questions to help reach the judgments regarding bias and applicability.

A. Risk of Bias
Describe the index test and how it was conducted and interpreted: The Hestia Criteria adresses clinical factors that may interfere with early-mortality. It includes 11 items regarding haemodynamically instability, necessity of thrombolysis or embolectomy, high risk for bleeding, need for oxygen supply to maintain oxygen saturation >90% >24 hour, pulmonary embolism diagnosed during anticoagulant treatment, intravenous pain medication >24 hour, medical or social reason for treatment in the hospital >24 hour, creatinine clearance of less than 30 ml/min, severe liver impairment, pregnancy, and documented history of heparininduced thrombocytopenia. If one of these items is fulfilled the patient cannot be classified into the low-risk category. There are some items that are, to some point subjective, and may be assessed differently by different clinicians. All data was retrieved from the electronic health record by trained study personnel blinded to the study outcome. Index test cut-offs were determined a priori and according to the test certification studies.
Were the index test results interpreted without knowledge of the results of the reference standard? Yes If a threshold was used, was it pre-specified? Yes Could the conduct or interpretation of the index test have introduced bias? Risk: Low

B. Concerns regarding applicability
Is there concern that the index, its conduct, or interpretation differ from the review question? Concern: Low TH Open © 2022. The Author(s).
Domain 2: Index Test -sPESI If more than one index test was used, please complete for each test.

A. Risk of Bias
Describe the index test and how it was conducted and interpreted: sPESI is a clinical stratification rule that takes into account some particular variables included in the PESI score. It attributes one point to the presence of each of the following variables: age above 80 years old, history of cancer, history of chronic cardiopulmonary disease, pulse >¼ 110 beats/min, systolic blood pressure <100mmHg, arterial oxyhemoglobin saturation level <90%. Patients were included in the low-risk category if none of these factors were present. All data was retrieved from the electronic health record by trained study personnel blinded to the study outcome. Index test cut-offs were determined a priori and according to the test certification studies.
Were the index test results interpreted without knowldege of the results of the reference standard? Yes If a threshold was used, was it pre-specified? Yes/Unclear Could the conduct or interpretation of the index test have introduced bias? Risk: Low

B. Concerns regarding applicability
Is there concern that the index, its conduct, or interpretation differ from the review question? Concern: Low Domain 2: Index Test -PESI If more than one index test was used, please complete for each test.

A. Risk of Bias
Describe the index test and how it was conducted and interpreted: PESI is a clinical stratification rule in which each risk variable is assigned a classification. Age above 80 years (one point for each year above), male sex (10 points), history of cancer (30 points), history of heart failure (10 points), history of chorinc lung disese (10 points), Pulse >¼ 110 beats/min (20 points), systolic blood pressure <100mmHg (30 points), respiratory rate >¼ 30 breaths/min (20 points), temperature <36°C (20 points), altered mental status (60 points), arterial oxyhemoglobin saturation level <90% (20 points). The added points result in a score. For this particular study, patients were defined as low-risk if this score was below 85 points. All data was retrieved from the electronic health record by trained study personnel blinded to the study outcome. Index test cut-offs were determined a priori and according to the test certification studies.
Were the index test results interpreted without knowldege of the results of the reference standard? Yes If a threshold was used, was it pre-specified? Yes/Unclear Could the conduct or interpretation of the index test have introduced bias? Risk: Low

B. Concerns regarding applicability
Is there concern that the index, its conduct, or interpretation differ from the review question? Concern: Low Domain 2: Index Test -IMPACT If more than one index test was used, please complete for each test.

A. Risk of Bias
Describe the index test and how it was conducted and interpreted: IMPACT is a stratification tool that is based on patients history and comorbidities to attribute a risk of early mortality in pulmonary embolism. This risk is estimated through a formula in which estimated % absolute risk A 30 day-mortality calculated risk below 1.5% was admitted to the low-risk category. All data was retrieved from the electronic health record by trained study personnel blinded to the study outcome. Index test cut-offs were determined a priori Were the index test results interpreted without knowldege of the results of the reference standard? Yes If a threshold was used, was it pre-specified? Yes/Unclear Could the conduct or interpretation of the index test have introduced bias? Risk: Low

B. Concerns regarding applicability
Is there concern that the index, its conduct, or interpretation differ from the review question? Concern: Low Domain 3: Reference Standard

A. Risk of Bias
Describe the reference standard and how it was conducted and interpreted: The reference standard is mortality. Information on patients mortality within the first 30 days after the acute event was retrieved from the Social Security Death Index.
Is the reference standard likely to correctly classify the target condition? Yes Were the reference standard results interpreted without knowledge of the results of the index test? Yes Could the reference standard, its conduct, or interpretation have introduce bias? Risk: Low

B. Concerns regarding applicability
Is there concern that the target condition as defined by the reference standard does not match the review question?
Concern: Low Domain 4: Flow and Timing

A. Risk of Bias
Describe any patients who did not receive the index test(s) and/or reference standard or who were excluded from the 2 Â 2 table (refer to flow diagram): From the patients included in the study, none was excluded from the index tests or mortality analysis.
Describe the time interval and any interventions between index test(s) and reference standard: Patients were managed through standard-of-care of that institution. Reference standard is assessed 30 days after the acute event, while the index tests were applied using information as close as possible to the pulmonary embolism initial presentation.
Since this was a retrospective study, the testing performed had no risk of bias. Patients (setting, intended use of index test, presentation, prior testing): Consecutive adult patients with objectively confirmed pulmonary embolism (included those with incidental discovery of the condition) The index tests aim to identify patients who are at low risk for early mortality, to consider early discharge and provide an anticoagulation strategy in an outpatient setting. The management of patients and decision regarding early discharge was in charge of the attending physician. The four centers of care didn't routinely use clinical stratification tools such as Hestia criteria, simplified Pulmonary Embolism Severity Index (sPESI) or Pulmonary Embolism Severity Index (PESI) for decision making. Hestia Criteria, sPESI and PESI were assessed by an investigator after patient inclusion in the study. Patients were also submitted to echocardiography or computed tomography pulmonary angiography to evaluate heart function (more particularly, to assess right ventricular dysfunction, an assumed prognostic factor). Some patients were submited to cardiac troponin testing. The index tests aim to identify patients who are at low risk for early mortality, to consider early discharge and provide an anticoagulation strategy in an outpatient setting. The management of patients and decision regarding early discharge was in charge of the attending physician. The four centers of care didn't routinely use clinical stratification tools such as Hestia criteria, simplified Pulmonary Embolism Severity Index (sPESI) or Pulmonary Embolism Severity Index (PESI) for decision making. Hestia Criteria, sPESI and PESI were assessed by an investigator after patient inclusion in the study.

A. Risk of Bias
Describe the index test and how it was conducted and interpreted: The Hestia Criteria adresses clinical factors that may interfere with early-mortality. It includes 11 items regarding haemodynamically instability, necessity of thrombolysis or embolectomy, high risk for bleeding, need for oxygen supply to maintain oxygen saturation >90% >24 hour, pulmonary embolism diagnosed during anticoagulant treatment, intravenous pain medication >24 hour, medical or social reason for treatment in the hospital >24 hour, creatinine clearance of less than 30 ml/min, severe liver impairment, pregnancy, and documented history of heparininduced thrombocytopenia. If one of these items is fulfilled the patient cannot be classified into the low-risk category.
There are some items that are, to some point subjective, and may be assessed differently by different clinicians. All data was retrieved from the electronic health record by trained study personnel blinded to the study outcome. Index test cut-offs were determined a priori and according to the test certification studies.
Were the index test results interpreted without knowledge of the results of the reference standard? Yes If a threshold was used, was it pre-specified? Yes Could the conduct or interpretation of the index test have introduced bias? Risk: Low

B. Concerns regarding applicability
Is there concern that the index, its conduct, or interpretation differ from the review question? Concern: Low Domain 2: Index Test -sPESI If more than one index test was used, please complete for each test.

A. Risk of Bias
Describe the index test and how it was conducted and interpreted: sPESI is a clinical stratification rule that takes into account some particular variables included in the PESI score. It attributes one point to the presence of each of the following variables: age above 80 years old, history of cancer, history of chronic cardiopulmonary disease, pulse >¼ 110 beats/min, systolic blood pressure <100mmHg, arterial oxyhemoglobin saturation level <90%. Patients were included in the low-risk category if none of these factors were present. All data was retrieved from the electronic health record by trained study personnel blinded to the study outcome. Index test cut-offs were determined a priori and according to the test certification studies.
Were the index test results interpreted without knowledge of the results of the reference standard? Yes If a threshold was used, was it pre-specified? Yes Could the conduct or interpretation of the index test have introduced bias? Risk: Low

B. Concerns regarding applicability
Is there concern that the index, its conduct, or interpretation differ from the review question? Concern: Low Domain 2: Index Test -PESI If more than one index test was used, please complete for each test.

A. Risk of Bias
Describe the index test and how it was conducted and interpreted: PESI is a clinical stratification rule in which each risk variable is assigned a classification. Age above 80 years (one point for each year above), male sex (10 points), history of cancer (30 points), history of heart failure (10 points), history of chorinc lung disese (10 points), Pulse >¼ 110 beats/min (20 points), systolic blood pressure <100mmHg (30 points), respiratory rate >¼ 30 breaths/min (20 points), temperature <36°C (20 points), altered mental status (60 points), arterial oxyhemoglobin saturation level <90% (20 points). The added points result in a score. For this particular study, patients were defined as low-risk if this score was below 85 points. All data was retrieved from the electronic health record by trained study personnel blinded to the study outcome. Index test cut-offs were determined a priori and according to the test certification studies.
Were the index test results interpreted without knowledge of the results of the reference standard? Yes If a threshold was used, was it pre-specified? Yes/Unclear Could the conduct or interpretation of the index test have introduced bias? Risk: Low

B. Concerns regarding applicability
Is there concern that the index, its conduct, or interpretation differ from the review question? Concern: Low TH Open © 2022. The Author(s).
Domain 2: Index Test -Clinical Gestalt If more than one index test was used, please complete for each test.

A. Risk of Bias
Describe the index test and how it was conducted and interpreted: This study compared clinical classification tools with clinical gestalt. The attending physician would be in charge of the management of the patients, and whether he would have early discharge to continue anticoagulation treatment in an outpatient setting. He made this decision based on patient history, clinical evaluation, the results of general blood tests, evaluation of right ventricular function, as patient's anticipated compliance.
Were the index test results interpreted without knowledge of the results of the reference standard?

Yes
If a threshold was used, was it pre-specified? There isn't a stabilified scoring system that allowed patient evaluation and stratification Could the conduct or interpretation of the index test have introduced bias?
Risk: Yes

B. Concerns regarding applicability
Is there concern that the index, its conduct, or interpretation differ from the review question?
Concern: Low

A. Risk of Bias
Describe the reference standard and how it was conducted and interpreted: The reference standard for early complications of pulmonary embolism are venous thromboembolism recurrence, major bleeding and all-cause death within 30 days. Recurrent pulmonary embolism was determined as the presence of a new intraluminal filling defect on computed tomographic angiography, or by a new perfusion scan defect involving >75% of a lung segment on a ventilation-perfusion scan. Recurrent deep vein thrombosis was determined as a new intraluminal fiilling defect on compression ultrasonography of symptomatic upper or lower limbs. Major bleeding was defined according to the Internal Society of Thrombosis and Haemosthasis definition: acute clinically overt bleeding associated with one or more of the following: decrease in hemoglobin of 2 g/dL or more; transfusion of 2 or more units of packed red blood cells; bleeding that occurs ins a critical site (instracranial/intra-spinal, intraocular, pericardial, intra-articular, intramuscular with compartment syndrome, or retroperitoneal); bleeding that is fatal, or bleeding that necessitates acute surgical intervention. All suspected adverse events and deaths were evaluated by a local adjudication committee.
Is the reference standard likely to correctly classify the target condition? Yes Were the reference standard results interpreted without knowledge of the results of the index test? Yes Could the reference standard, its conduct, or interpretation have introduce bias? Risk: Low

B. Concerns regarding applicability
Is there concern that the target condition as defined by the reference standard does not match the review question?
Concern: Low

A. Risk of Bias
Describe any patients who did not receive the index test(s) and/or reference standard or who were excluded from the 2 Â 2 table (refer to flow diagram): From the patients who met criteria to be included in the study, 10 were excluded because they denied consent, and 5 were excluded by inability to be followed up.
Describe the time interval and any interventions between index test(s) and reference standard: Index tests were evaluated by a local investigator following the patient admission in te study. It did not interfere with the decision to early discharge and was performed before the appearance of adverse events or death (the reference standard) For the diagnosis of the pulmonary embolism were used ventilation-perfusion lung scan, or computed tomography pulmonary angiography. On the event of the previous testing were non-diagnostic, compression ultrasonography of the lower extremities was performed. The index tests aim to identify patients who are at low risk for early mortality, to consider early discharge and provide an anticoagulation strategy in an outpatient setting.
Index test(s): Hestia criteria, simplified Pulmonary Embolism Severity Index (sPESI) Reference standard and target condition: The reference standard for mortality within the first 30 days of pulmonary embolism presentation is mortality itself. For the diagnosis of the pulmonary embolism were used ventilation-perfusion lung scan, or computed tomography pulmonary angiography. On the event of the previous testing were non-diagnostic, compression ultrasonography of the lower extremities was performed. The index tests aim to identify patients who are at low risk for early mortality, to consider early discharge and provide an anticoagulation strategy in an outpatient setting.
Is there concern that the included patients do not match the review question? Concern: Low Domain 2: Index Test -Hestia Criteria

A. Risk of Bias
Describe the index test and how it was conducted and interpreted: The Hestia Criteria adresses clinical factors that may interfere with early-mortality. It includes 11 items regarding haemodynamically instability, necessity of thrombolysis or embolectomy, high risk for bleeding, need for oxygen supply to maintain oxygen saturation >90% >24 hour, pulmonary embolism diagnosed during anticoagulant treatment, intravenous pain medication >24 hour, medical or social reason for treatment in the hospital >24 hour, creatinine clearance of less than 30 ml/min, severe liver impairment, pregnancy, and documented history of heparininduced thrombocytopenia. If one of these items is fulfilled the patient cannot be classified into the low-risk category.
There are some items that are, to some point subjective, and may be assessed differently by different clinicians. Index test cut-offs were determined a priori and according to the test certification studies.
Were the index test results interpreted without knowledge of the results of the reference standard? Yes If a threshold was used, was it pre-specified? Yes Could the conduct or interpretation of the index test have introduced bias? Risk: Low

B. Concerns regarding applicability
Is there concern that the index, its conduct, or interpretation differ from the review question? Concern: Low Domain 2: Index Test -sPESI If more than one index test was used, please complete for each test.

A. Risk of Bias
Describe the index test and how it was conducted and interpreted: sPESI is a clinical stratification rule that takes into account some particular variables included in the PESI score. It attributes one point to the presence of each of the following variables: age above 80 years old, history of cancer, history of chronic cardiopulmonary disease, pulse >¼ 110 beats/min, systolic blood pressure <100mmHg, arterial oxyhemoglobin saturation level <90%. Patients were included in the low-risk category if none of these factors were present. Index test cut-offs were determined a priori and according to the test certification studies.
Were the index test results interpreted without knowldege of the results of the reference standard? Yes If a threshold was used, was it pre-specified? Yes/Unclear Could the conduct or interpretation of the index test have introduced bias? Risk: Low

B. Concerns regarding applicability
Is there concern that the index, its conduct, or interpretation differ from the review question? Concern: Low Domain 3: Reference Standard

A. Risk of Bias
Describe the reference standard and how it was conducted and interpreted: The reference standard is mortality in the period 30 days after the onset of acute pulmonary embolism. This information was retrieved using patient or proxy interviews, and/or hospital chart review.
Is the reference standard likely to correctly classify the target condition? Yes Were the reference standard results interpreted without knowledge of the results of the index test? Yes Could the reference standard, its conduct, or interpretation have introduce bias? Risk: Low

B. Concerns regarding applicability
Is there concern that the target condition as defined by the reference standard does not match the review question?