Comment and Update on “Using Artificial Intelligence to Manage Thrombosis Research, Diagnosis, and Clinical Management”

We recently read the interesting article entitled: “Using Artificial Intelligence to Manage Thrombosis Research, Diagnosis, and Clinical Management” by Mishra and Ashraf.[1] In this paper, the authors reviewed the most recent application of several artificial intelligence (AI) methods for thromboembolism prediction.

The authors started their paper explaining AI theory and the potential of AI methods in compiling observational data and recognizing patterns hidden behind large amount of data, which can be impossible for the human brain to process. Then, they reviewed some important studies concerning the prediction of myocardial infarction, the therapeutic dose estimation, and clinical application of robotics in thrombosis. The authors concluded their analysis by presenting some limitations and perspectives of AI application on clinical data.

The intent of this letter is to update the readership of this journal on applications of AI in thrombosis prediction, and thus extends on the report of Mishra and Ashraf.[1]

Mishra and Ashraf[1] referenced a study performed by our group entitled: “Artificial neural network for prediction of venous thrombosis recurrence.” This study was presented at the American Society of Hematology Annual Meeting at 2016.[2] It was the first report in the literature regarding an artificial neural network (ANN) model to predict the recurrence of venous thromboembolism (RVTE). We considered 44 independent factors as inputs, which included patient and clinical data, and the results showed that it was possible to obtain an accurate ANN model to predict RVTE, also highlighting that this was a promising field to explore.

Over the last years, we have improved this model and obtained a more comprehensive view of the ANN potential for its use in such predictions. The first observation was the high number of inputs, which could make the use of such ANN difficult in practice. Thus, we performed a study to identify which inputs represented the most important factors to predict RVTE through principal component analysis (PCA).[3] In this paper, we reported that among all the independent factors considered, 18 are sufficient to mathematically predict RVTE. The important factors included blood cells, lipids, coagulation protein levels, age, as well as previous VTE parameters. This was an important step, since it showed that parameters that are simple to obtain could be promising in the development of new prediction models.

In an upcoming issue of International Journal of Medical Informatics, we proposed an association between PCA and ANN modeling which is not commonly used in health studies. Also, we presented three new ANN models for RVTE prediction.[4] The difference among the three models relies on its inputs: ANN 1: all the 39 factors considered in the previous study[3]; ANN 2: the 18 most important factors determined by PCA[3]; ANN 3: 15 factors combining PCA results and practical aspects, which excluded anticoagulant treatment duration, protein C, protein S, and antithrombin from the inputs of ANN 2, and included the factor: provoked/unprovoked. The development of the third model aimed to obtain the most simplified model possible to predict RVTE, which would make its application much easier. We performed a comprehensive search for the best ANN model, including comparison among learning optimization algorithms, number of hidden layers, and number of units in each hidden layer. We also performed another importance analysis of each input factor by using a weight method.[5]

The criterion to choose the best model was higher accuracy, area under the curve, fivefold cross-validation performance, and layer saturation—which is very important since it is a direct measurement of the ANN generalization capacity. The results showed that it was possible to obtain an accurate model for each input set, with accuracies above 92%. The third and simplest ANN model presented an accuracy of 97% for validation + test dataset, and the cross-validation runs showed that its prediction capability was not affected by considering different training dataset. This study showed that the association of multivariate techniques and AI is very promising and can be explored to develop new models in healthy studies.

Recently, we also reported another study to identify the effect of independent factors on RVTE by using designs of experiments (DoE) techniques.[6] DoE is a statistic tool that performs sensitivity analysis of input variables or of model parameters. It is an interesting technique that does not require new observational data and can be applied by using fitted mathematical models. In this work, the ANN 1 from the previous work was used[4] (i.e., considered all the input factors) to compute the effect of each variable.

When analyzing the results obtained by the three techniques (PCA,[3] weight analysis,[4] DoE[6]), we observed that several factors appeared repeatedly. White blood cells, red blood cells, age, protein S, and D-dimer were identified as important by all three techniques. This is an important outcome, since it shows that an unsupervised method, such as PCA, can be a reliable tool for preprocessing inputs before the development of new predictive methods.

Despite the new results obtained by our group, some interesting papers published in later 2019 and 2020 showed new important conclusions, which should be highlighted. [Table 1] summarizes the most important features of each new paper published in the field.

Table 1 Summary results of new papers on artificial intelligence and thrombosis

Author(s)	Collected data	Focus of the study	Artificial intelligence technique	Summary results
Nafee et al[7]	APEX trial[8]	Identification of acutely ill patients at high risk for venous thromboembolism	Super learner model	The proposed models presented with similar performance and were more accurate than the IMPROVE score.[9]
Wang et al[10]	Chinese patients	VTE risk prediction among Chinese patients	Eight different methods	Random forest achieved the best performance. However, it presented lower sensitivity when compared with the Padua score.[11]
Nudel et al[12]	Surgery patients	Predict gastrointestinal leak and venous thromboembolism after weight loss surgery	ANN.  Gradient boosting machines (XGBs).  Logistic regression.	XGBs were better than the other methods. When comparing with the BariClot score,[13] both AI models were also much superior.
Tajik et al[14]	International Warfarin Pharmacogenetics Consortium (IWPC) data	Body mass index (BMI) influence on other VTE risk factors	Eight different methods	Random Forest presented the highest prediction accuracy. The authors observed that the order of importance of each factor is different for each BMI group (normal, over-weight, obesity, and morbidly obesity) and that this should be taken in account when analyzing VTE risk.

Abbreviations: AI, artificial intelligence; ANN, artificial neural networks; APEX, The Acute Medically Ill VTE (venous thromboembolism) prevention with Extended Duration Betrixaban Trial; BMI, body mass index; IMPROVE, International Medical Prevention Registry on Venous Thromboembolism Score; VTE, venous thromboembolism.

When comparing all those new studies, it is important to mention that all still lack external validation using independent and multicentric datasets. Thus, we endorse that large amounts of data should be collected to understand the influence of the sample size in the obtained models. We also suggest to evaluate the quality of the collected dataset on the models’ results, which Wells[15] stated as one of the drawbacks of current health information systems. Obtaining larger datasets could improve the AI methods by stratifying populations according to important risk factors as done by Tajik et al.[14] Oriented models for patients with different sex (men/women), antiphospholipid syndrome, cancer, and for those diagnosed with SARS-Cov-2 (which is now known as an important provoked risk factor[16] [17] [18]), could improve disease prophylaxis and patient recovery. In summary, we believe that AI methods are important for the development of new clinical decision support systems, and mathematicians and clinicians should embrace the potential of AI in health care.

Publication History

Article published online:
01 February 2021

• References

• 1 Mishra A, Ashraf MZ. Using artificial intelligence to manage thrombosis research, diagnosis, and clinical management. Semin Thromb Hemost 2020; 46 (04) 410-418
• 2 Romano AVC, Martins TD, Maciel R, De Paula EV, Annichino-Bizzacchi JM. Artificial neural network for prediction of venous thrombosis recurrence. Blood 2016; 128 (22) 3771
• 3 Martins TD, Annichino-Bizzacchi JM, Romano AVC, Filho RM. Principal component analysis on recurrent venous thromboembolism. Clin Appl Thromb Hemost 2019; 25: 1076029619895323
• 4 Martins TD, Annichino-Bizzacchi JM, Romano AVC, Maciel Filho R. Artificial neural networks for prediction of recurrent venous thromboembolism. Int J Med Inform 2020; 141: 104221
• 5 O'Donoghue J, Roantree M, McCarren A. Detecting feature interactions in agricultural trade data using a deep neural network. In: Bellatreche L, Chakravarthy S. eds. Big Data Analytics and Knowledge Discovery. DaWaK 2017. Lecture Notes in Computer Science.. Lyon, France: Springer; 2017: 449-458
• 6 Ottaiano GY, Annichino-Bizzacchi JM, Maciel-Filho R, Martins TD. Determining predictive factors of venous thromboembolism using design of experiments, Paper presented at: ISTH 2020 Virual Congress: Thrombosis and Hemostasis Conference, July 12–14. 2020 ; Milan
• 7 Nafee T, Gibson CM, Travis R. et al. Machine learning to predict venous thrombosis in acutely ill medical patients. Res Pract Thromb Haemost 2020; 4 (02) 230-237
• 8 Cohen AT, Harrington R, Goldhaber SZ. et al. The design and rationale for the acute medically Ill venous thromboembolism prevention with extended duration betrixaban (APEX) study. Am Heart J 2014; 167 (03) 335-341
• 9 Rosenberg D, Eichorn A, Alarcon M, McCullagh L, McGinn T, Spyropoulos AC. External validation of the risk assessment model of the International Medical Prevention Registry on Venous Thromboembolism (IMPROVE) for medical patients in a tertiary health system. J Am Heart Assoc 2014; 3 (06) e001152
• 10 Wang X, Yang YQ, Liu SH, Hong XY, Sun XF, Shi JH. Comparing different venous thromboembolism risk assessment machine learning models in Chinese patients. J Eval Clin Pract 2020; 26 (01) 26-34
• 11 Barbar S, Noventa F, Rossetto V. et al. A risk assessment model for the identification of hospitalized medical patients at risk for venous thromboembolism: the Padua Prediction score. J Thromb Haemost 2010; 8 (11) 2450-2457
• 12 Nudel J, Bishara AM, de Geus SWL. et al. Development and validation of machine learning models to predict gastrointestinal leak and venous thromboembolism after weight loss surgery: an analysis of the MBSAQIP database. Surg Endosc 2020; (e-pub ahead of print). DOI: 10.1007/s00464-020-07378-x.
• 13 Dang JT, Switzer N, Delisle M. et al. Predicting venous thromboembolism following laparoscopic bariatric surgery: development of the BariClot tool using the MBSAQIP database. Surg Endosc 2019; 33 (03) 821-831
• 14 Tajik F, Wang M, Zhang X, Han J. Evaluation of the impact of body mass index on venous thromboembolism risk factors. PLoS One 2020; 15 (07) e0235007
• 15 Wells P. Predictive analytics by deep machine learning: a call for next-gen tools to improve health care. Res Pract Thromb Haemost 2020; 4 (02) 181-182
• 16 Pavoni V, Gianesello L, Pazzi M, Stera C, Meconi T, Frigieri FC. Evaluation of coagulation function by rotation thromboelastometry in critically ill patients with severe COVID-19 pneumonia. J Thromb Thrombolysis 2020; 50 (02) 281-286
• 17 Iba T, Levy JH, Levi M, Thachil J. Coagulopathy in COVID-19. J Thromb Haemost 2020; 00: 1-7
• 18 Engelen MM, Vanassche T, Balthazar T. et al. Incidence of venous thromboembolism in patients discharged after COVID-19 hospitalisation, Paper presented at: ISTH 2020 Virtual Congress: Thrombosis and Hemostasis Conference, July 12–14. 2020 ; Milan