Patient-Completed Caprini Risk Score for Venous Thromboembolism Risk Assessment: Developed and Validated from 1,017 Medical and Surgical Patients

Background  The Caprini Risk Score (CRS) is one of the most widely used risk assessment models for venous thromboembolism (VTE). A well-validated patient-completed CRS form may allow patients to self-report and simplify the evaluation by health care workers. Methods  The Chinese version of the CRS was optimized for easy understanding in a pilot study. The amended CRS form was completed by prospectively recruited patients and blinded nurses. The agreement levels of the individual questions and the total scores of patient and nurse-completed forms were compared using the Kappa value. The total scores were used for risk stratification of patients. Correlation and differences between patient and nurse-completed forms were analyzed using the Spearman correlation and Bland–Altman method, respectively. Results  We recruited 504 medical patients and 513 surgical patients, aged 52.7 ± 16.3 years, of which 443 (43.6%) were men, and 91.6% of the patients were educated beyond junior high school. The patients spent less time to complete the form compared with trained nurses. There was good question-to-question agreement between patient and nurse-completed CRS ( k >0.6 for most questions, p  < 0.0001). The total scores also showed good agreement ( k  = 0.6097, p  < 0.0001), and enabled the classification of patients into different risk groups. The patient and nurse-derived scores were highly correlated (Spearman's r  = 0.84), and without extreme values ( p  < 0.0001). Conclusion  We have created and verified a Chinese version of the patient-completed CRS, which showed good agreement and correlation with nurse-completed CRS. CRS represents a suitable tool for VTE risk assessment of hospitalized patients in China.


Introduction
Venous thromboembolism (VTE), including pulmonary thromboembolism (PTE) and deep venous thrombosis (DVT), is a common postoperative complication and a major cause of unexpected death in hospitalized patients. 1 Nearly half of the VTE cases are associated with hospitalization, which is a major triggering factor for VTE. 2,3 VTE is the cause of death in up to 2.1% of hospitalized patients in China. 4 Approximately 20 to 50% of patients who survive VTE develop post-thrombotic syndrome, which may lead to venous ulcers, and 2 to 4% of patients with PTE develop chronic thromboembolic pulmonary hypertension (CTEPH), which is potentially fatal. 5 These conditions have a negative effect on the quality of life of individuals and heavily burden the health care system. 6 Therefore, the appropriate prophylaxis of VTE is highly important for the safety of hospitalized patients and for avoiding the sequelae of VTE. 7 The Caprini Risk Score (CRS) is the most widely used risk assessment model (RAM) for VTE in clinical practice and has been validated in medical patients and surgical patients worldwide. 8,9 The ninth edition of the American College of Chest Physicians (ACCP) Antithrombotic Therapy and Prevention of Thrombosis guidelines has recognized that the Caprini RAM can be used for stratification of non-orthopaedic surgical patients. 10 However, the current CRS assessment is mainly completed by health care workers such as doctors and nurses, which is accurate, but may be time consuming. 11 To increase the level of knowledge, involvement, and compliance of patients, 12,13 a patient completed CRS in English was created in 2017. 14 Thus far, the patientcompleted CRS has been translated into Spanish, Polish, and Arabic, and its effectiveness has been verified in those versions. 15 The Chinese version of the patient-completed CRS was constructed earlier but was verified only in a small group of patients. 16 It lacks external validation and application in clinical practice. Therefore, the present study aimed to test the reliability and validity of the Chinese version of patientcompleted CRS in a large number of hospitalized patients and to evaluate whether this RAM can be widely used in clinical practice to replace the assessment by health care workers.

Study Design
We conducted a prospective, observational, single-center study and recruited 514 inpatients from the surgical units and 503 inpatients from the medical unit between January 1 and December 31, 2019. The inclusion criteria were as follows: (1) patients older than 18 years; (2) patients with first admission to the medical and surgical units within 24 hours; and (3) willingness to participate in the study. The exclusion criteria were as follows: (1) patients with a diagnosis of PTE or DVT on admission; (2) patients with cognitive impairment; (3) inability to read or write; (4) patients with mental diseases; (5) patients with visual impairment; and (6) refusal to cooperate. Our work has been reported in line with the STARD (Standards for the Reporting of Diagnostic accuracy studies) criteria.

Construction and Validation of the Patient-Completed CRS
The English patient-completed CRS was translated into Chinese by two independent experts with good English proficiency and professional medical knowledge. Then, the translated version was reviewed by three experts with rich clinical experience in VTE to ensure its accuracy.
To identify the ambiguous and misinterpreted areas of the Chinese version, a pilot experiment was conducted, in which 100 patients admitted to the Department of Pulmonary and Critical Care Medicine in our hospital were consecutively recruited using the inclusion and exclusion criteria. The patients were asked to complete the form and provide feedback, which were used to adjust the expression of the context and the order of questions to help Chinese patients understand the form better (►Supplementary Fig. S1).
Patients were prospectively recruited from internal medicine and surgery departments by a continuous inclusion and convenience sampling method. The patients and trained nurses were asked to complete the revised Chinese version of the CRS form 24 hours after admission to hospital. The patients did not undergo any intervention during their evaluation process (NCT04663477).

Statistical Analysis
Agreement between patient-completed CRS and trained nurse-completed CRS was measured using Cohen's Kappa compared with trained nurses. There was good question-to-question agreement between patient and nurse-completed CRS (k >0.6 for most questions, p < 0.0001). The total scores also showed good agreement (k ¼ 0.6097, p < 0.0001), and enabled the classification of patients into different risk groups. The patient and nurse-derived scores were highly correlated (Spearman's r ¼ 0.84), and without extreme values (p < 0.0001). Conclusion We have created and verified a Chinese version of the patient-completed CRS, which showed good agreement and correlation with nurse-completed CRS. CRS represents a suitable tool for VTE risk assessment of hospitalized patients in China.
value. The scores were summed up and Spearman's coefficient was used to assess the overall correlation between patient and nurse-derived total scores. The Bland-Altman method was used to evaluate the differences between the scores calculated by the patients and those by the nurses. The overall CRS was categorized into very low (0), low (1-2), moderate (3)(4), and high risk (!5) based on the ninth ACCP guidelines, and the agreement level between patient-completed and nurse-completed CRS was measured using Kappa values. All statistical analyses were conducted using SAS 9.4 (SAS Institute Inc., Cary, North Carolina, United States). Data were expressed as the composition ratio and mean AE standard deviation (SD). All p-values were calculated using twosided tests, and significance was determined a priori at the 0.05 level.

Characteristics of the Subjects
In total, we recruited 1,017 patients to complete the CRS. Among them, 504 were medical patients and 513 were surgical patients. Mean (AE SD) patient age was 52.7 AE 16.3 years, and 443 (43.6%) were men. Most of the participants were well educated and lived in the city (►Table 1). Medical patients were from pulmonary critical care medicine, gastroenterology, cardiology, rheumatology, neurology and surgical patients were from gynecology, orthopaedics, general surgery, etc. (►Table 2). The median time for patients to complete the form was 4 minutes (3-6 minutes), which was less than the time needed by trained nurses.

Question-to-Question Agreement between Patient-Completed and Nurse-Completed CRS
Cohen's Kappa values were used to measure the extent of agreement between patients' and health care providers' assessments. The vast majority of the items of the scale showed good agreement between the two groups, except for "one surgery-related item," and the Cohen's Kappa values ranged from 0.63 to 1.0. The corresponding data are presented in ►Table 3. High inter-rater reliability was found for all items, and the Kappa values for all items were statistically significant (all p-values <0.0001). Furthermore, the items on the evaluation scale showed consistency between the patient-evaluated and nurse-evaluated scores. The Kappa values are all statistically significant after testing (p-value <0.0001) (►Table 3). The degree of agreement was evaluated based on the Kappa value (Kappa 0.4 indicated low consistency, 0.4< Kappa 0.6 indicated moderate consistency, 0.6< Kappa 0.8 indicated high consistency, 0.8 <Kappa 1 indicated high consistency). The vast majority of items showed Kappa values> 0.6, except for elective surgery.

Agreement Level of Risk Stratification between Patient-Completed and Nurse-Completed CRS
We added up the patient-completed and nurse-completed CRS and categorized them into very low, low, medium, and high risk according to the ninth ACCP guidelines. We found that there were 18 (1.8%) patients with very low risk, 152 (15.0%) patients with low risk, 242 (23.8%) patients with medium, and 339 (33.3%) patients with high risk in both patient-completed and nurse-completed risk stratification. The Kappa value was 0.6097 (p < 0.0001), as shown in ►Table 4 and ►Fig. 1. The Kappa value was 0.6097 (p < 0.0001), indicating that the risk stratification of patient self-evaluation and nurse-completed evaluation was consistent.

Correlation and Difference between the Total Scores Calculated by Patient-Completed and Nurse-Completed CRS
The Spearman correlation and the Bland-Altman method were used to measure the correlation and difference between patient-completed and nurse-completed CRS. The Spearman correlation coefficient was 0.8415 (p < 0.0001) (►Fig. 2), and we did not find any trend for extreme values according to the Bland-Altman (►Fig. 3). There is no difference between the scores of patients' self-assessment and nurses' assessment.

Discussion
We created and validated the Chinese version of CRS, which is the most useful VTE RAM, and showed good correlation and agreement between patient-filled and nurse-filled forms. Our study may facilitate the use of the VTE RAM in China and promote the engagement of patients in the prevention of thrombosis.
Most clinical evaluations are performed by medical staffs, which are accurate but time-consuming. For this reason it has been suggested to limit the use of these scoring models in the clinical setting. 11 Additionally, this limits the involvement of patients in the assessment process. In contrast, patient-reported questionnaires have been found effective and well suited when complex health issues are explored in medical and surgical patients. 17,18 CRS is the most widely used risk assessment tool and has been approved for risk-stratification, including very lowrisk, low-risk, moderate-risk and high-risk categories, according to the ninth ACCP guidelines. 9 High-risk patients are individuals with scores above 5, indicating a sub-group of patients at great risk for VTE. The high-risk patients may need preventive anticoagulant treatment. According to the guidelines, 9 the set point of high risk is 5, but this has since been found to vary by study population. Specifically, the set point of highest risk is 9 for patients having general surgery, 19 10 for patients with total joint arthroplasty, 20 11 for patients undergoing colorectal cancer surgery, 21 and 12 for patients following hip fracture. 22 In our study, nearly 27% of participants scoring above 5 were categorized as high risk. More importantly, as the VTE risk escalates with each point increase, we reasoned that there may not be a fixed and universal set point in VTE risk assessment. Therefore, we believe that it would be informative to report the number of patients different scores. The numbers (%) of patients with score above 5, 6, 7, 8, 9,  Chen et al 16 tested a Chinese version of the patient-completed CRS in a small group of patients and found good correlation and agreement levels. However, in 2019, Veith et al 23 compared patient-completed and physician-completed CRS in plastic surgery patients and found that reliance on patientcompleted scores alone would promote nearly 25% of patients receiving inappropriate prophylactic strategies. In their study, almost 50% of the risk factors showed disagreement between patient-and health provider-completed forms (►Table 5). Taken together, these studies show that whether the CRS can be completed by patients instead of health care workers is controversial, and further external validation using a larger number of patients is needed in for the Chinese version. Therefore, we created and validated the Chinese patient-completed CRS according to the guidelines of the International Society for Pharmacoeconomics and Outcome Research (ISPOR). According to the results of our pilot study and original authors' suggestions for revision, we modified sections 4.1, 5.1-5.3, 7.1, and 7.3 of the questionnaires to help Chinese patients understand it easily.
We measured the question-to-question agreement level between patient-completed and nurse-completed CRS. Our results showed that there was good agreement in all questions except one: "I have a scheduled surgery under general or regional anesthesia for less than 45 minutes." The Kappa value of that question was 0.4285, while Kappa values of all other questions ranged from 0.6036 to 1.000 (all p-values <0.0001), which is similar to the range reported by Fuentes et al 14 We think that this may be related to the lack of  professional medical knowledge of patients so that they were likely to inaccurately estimate the length of the operation time. Hence, according to our study, we indicated that trained nurses should assess the question with patients to determine the accuracy of that question. It is worth noting that our study showed good agreement for the question "age" and the sub-questions of "For women only, add 1 point for each statement that applies to you," the Kappa values of these questions were nearly 1.000 and 0.8786, 0.9226 and 0.7997, indicating that medical staff did not need to assess these questions again. It is essential to evaluate the risk of thrombosis of patients and select optimal prophylactic strategy. 24 Table 3 Question-to-question agreement level between patient-completed and nurse-completed forms (n ¼ 1,017) Our study showed good agreement in risk stratification according to the total scores calculated by patient and nurse-filled form, which can help clinicians select correct anticoagulation methods. The strength of our study was that our study did an external validation of Chinese patient-completed CRS, and so far our study was the largest and the most convincing. We conducted a pilot experiment and optimized the CRS using the results obtained from 100 patients. Then we continuously enrolled a large number of patients from internal medicine and surgery departments. The limitation of this research was that this was a prospective, observational, single-center study; in the future, we plan to conduct a multicenter study. Additionally, our patients are generally highly educated, as nearly half of them had a college degree or higher, which may affect the usefulness of the form in less well-educated patients.
In conclusion, we adapted and revised the English version of the patient-completed CRS into Chinese and validated its effectiveness in a large group of patients. Our study showed that there is a high agreement level and good correlation between patient-completed and trained nurse-completed CRS. We conclude that the Chinese patient-completed CRS is a suitable tool for VTE risk assessment in hospitalized patients in China.
1. Caprini Risk Score (CRS) is usually completed by health workers in a time-consuming process. 2. English version of patient-completed CRS has been validated to have good agreement with physician-completed CRS.  Fig. 1 Agreement level in risk stratification between nurse-completed and patient-completed forms (n ¼ 1017). The figure indicated that the risk stratification of patient self-evaluation and nurse-completed evaluation was consistent. Fig. 2 Correlation between patient-completed and nurse-completed Caprini risk score (n ¼ 1017). The Spearman correlation coefficient was 0.8415. There is no difference between the scores of patients' self-assessment and nurses' assessment.   What Does This Paper Add?

Authors' Contributions
A.S. and Z.Z. had full access to all data in the study and take responsibility for the content of the manuscript. A.S. and Z.Z. conceived and designed this research. Z.Z. and Y.W. revised it. Q.L. and F.D. integrated data and analyzed data. Y.W. and Z.Z. wrote the manuscript. W.P., K.Z., J.W., W. X., W.W., and P.Y. contributed to the interpretation of the data and clinic inputs. All authors approved the final version.

Ethical Approval
The study was conducted with informed consent from all participants and approval from the Local Ethics Committees (No. 2017-24).

Ethics Approval and Consent to Participate
All participants provided informed consent, and the Local Ethics Committees approved the study (No. 2017-24).

Patient Consent for Publication
All included patients informed consent form and consented to publish the study.

Patient Consent for Publication
All patients consented to publish the study.

Provenance and Peer Review
Not commissioned, externally peer-reviewed.

Availability of Data and Materials
The datasets used and/or analyzed during the current study are available from the corresponding authors upon reasonable request.

Conflict of Interest
None declared.