Computer Vision Technology in the Differential Diagnosis of Cushing’s Syndrome

Abstract

Objective Cushing’s syndrome is a rare disease characterized by clinical features that show morphological similarity with the metabolic syndrome. Distinguishing these diseases in clinical practice is challenging. We have previously shown that computer vision technology can be a potentially useful diagnostic tool in Cushing’s syndrome. In this follow-up study, we addressed the described problem by increasing the sample size and including controls matched by body mass index.

Methods We enrolled 82 patients (22 male, 60 female) and 98 control subjects (32 male, 66 female) matched by age, gender and body-mass-index. The control group consisted of patients with initially suspected, but biochemically excluded Cushing’s syndrome. Standardized frontal and profile facial digital photographs were acquired. The images were analyzed using specialized computer vision and classification software. A grid of nodes was semi-automatically placed on disease-relevant facial structures for analysis of texture and geometry. Classification accuracy was calculated using a leave-one-out cross-validation procedure with a maximum likelihood classifier.

Results The overall correct classification rates were 10/22 (45.5%) for male patients and 26/32 (81.3%) for male controls, and 34/60 (56.7%) for female patients and 43/66 (65.2%) for female controls. In subgroup analyses, correct classification rates were higher for iatrogenic than for endogenous Cushing’s syndrome.

Conclusion Regarding the advanced problem of detecting Cushing’s syndrome within a study sample matched by body mass index, we found moderate classification accuracy by facial image analysis. Classification accuracy is most likely higher in a larger sample with healthy control subjects. Further studies might pursue a more advanced analysis and classification algorithm.

Key words

pituitary adenoma - glucocorticoids - metabolic features, obesity

Introduction

Cushing’s syndrome (CS) is a disease caused by endogenous or iatrogenic hypercortisolism. It leads to metabolic complications, neuropsychiatric impairment and typical morphological changes such as central obesity, facial plethora, acne, proximal myopathy and cervical fat pads. Increased morbidity and mortality can be normalized with early diagnosis and treatment [1] [2] [3] [4] [5]. However, the diagnosis is still established with an assumed delay of 2–6 years, resulting in the need for new approaches to screening and diagnosis [6] [7].

Many clinical and biochemical features of CS are unspecific and can also be found in patients suffering from the metabolic syndrome (MetS) which consists of abdominal obesity, hypertension, impaired glucose metabolism and dyslipidemia [1] [8]. In MetS, obesity can lead to a similar phenotype to that observed in CS patients. Additionally, some biochemical tests for CS can be falsely positive in the presence of MetS which features proinflammatory activity [9] [10] [11]. Distinguishing patients suffering from CS, especially at a subclinical stage, from patients exhibiting similar features without true endogenous hypercortisolism thus presents a challenge in clinical practice [1] [6] [12].

Computer vision technology has recently been found to be a promising diagnostic tool for diseases that feature morphological changes of the head and face, such as acromegaly or dysmorphic syndromes [13] [14]. In a pilot study from 2013, we reported promising results regarding the use of computer vision technology for the detection of CS in a small and selected female-only sample [15]. In this follow-up study, we have tried to address the advanced problem of distinguishing CS from MetS subjects, thereby exploring a more realistic setting for clinical applications of the method. For this, we have (I) enrolled a much larger number of subjects including males, (II) included a control group matched by body mass index (BMI), and (III) have used an improved set of facial landmarks for computer vision and classification based on other research from our group [16] [17].

Subjects and Methods

We performed a multicenter diagnostic study in a cross-sectional design. Data was collected from January 2013 to November 2015. The study was approved by the Ethics Committee of Ludwig Maximilians University Munich and conforms to the Declaration of Helsinki. All subjects gave written informed consent for participation in this study.

Patients and Controls

Patients

Out of 250 screened subjects, 82 fulfilled all inclusion criteria and were included in the study. Patients were recruited from the endocrine (n=53) and rheumatologic (n=4) clinics of LMU Munich, Max Planck Institute of Psychiatry in Munich (n=14), Würzburg University Hospital (n=10), and Endocrinology in Berlin-Charlottenburg (n=1). We defined the following inclusion criteria for patients: Age> 18 years, BMI 20–50 kg/m²; excluded pregnancy; at least 2 abnormal results among the following tests: midnight salivary cortisol, 1 mg dexamethasone suppression test and 24-h urinary cortisol; oral glucocorticoid treatment with a Prednisone equivalent greater than 0.17 mg/kg of body weight/day for longer than 3 months; no more than 3 months post successful transsphenoidal surgery or adrenalectomy.

Control group

Control subjects were recruited at LMU Munich from a sample of patients that presented with suspected CS for further diagnostic work-up, but ultimately negative test results. For example, symptoms of these patients were components of MetS like abdominal obesity, or facial features like plethora or steroid acne. CS had to be biochemically excluded by normal results in at least one of the following tests: midnight salivary cortisol, 1 mg dexamethasone suppression test or 24-h urinary cortisol. BMI had to be between 20 and 50 kg/m². Out of 148 screened subjects, we included 98 subjects who could be matched to patients by age, gender and BMI. All patients and controls were Caucasian.

Acquisition of facial photographs

Frontal and profile pictures of the head and face were acquired using a regular digital camera (Canon IXUS 125 HS). The procedure was standardized regarding homogenous lighting, neutral facial expression, uniformly white or bright grey background and removal of eyeglasses or hair covering the face. Pictures were then processed in a standardized way, which included resizing to 1500×1000 pixels, renaming for pseudonymization, and import into the computer application FIDA (Facial Image Diagnostic Aid, Würtz/Günther, Ruhr-University Bochum).

Image classification

A face graph consisting of a grid of 52 nodes for the frontal view and 36 nodes for the profile view were semi-automatically placed on the digital photographs covering disease-relevant predefined landmarks, for example cheeks for plethora and facial margins for width ([Fig. 1]). The graph differs from the one previously used, which covered the entire head. The currently used landmarks only cover the face and no longer extend beyond the hairline [16] [17]. The labeling procedure was performed with knowledge of the diagnosis. The application FIDA automatically classified the subjects into two categories (syndrome/control). Image classification is based on analysis of texture and geometry within the described grid of semi-automatically placed nodes covering relevant landmarks. The analysis of texture relies on the calculation of similarity functions for Gabor jets at the nodes. A Gabor jet is a vector that describes the image’s texture information around the node [18]. The analysis of geometry is based on comparison of the relative locations of the nodes. As in the previous study we used the functions P (scalar product including phase) for texture and L (edge difference length) for geometry analysis [15]. The data were evaluated using a leave-one-out cross-validation procedure, in which every subject is once excluded from the training set and then tested, thus giving a valid reading for individual subjects. The images are classified according to similarities with either group (syndrome/control) from the training database using a maximum likelihood classifier. Subjects are assigned to the diagnostic group that they share more similarities with, expressed as mathematical functions obtained from image analysis.

Fig. 1 Frontal and profile views of a rendered model for illustration of the method. The face graph, consisting of connected nodes placed on relevant facial landmarks, is for used for image analysis and classification.

Statistical analyses

To compare age and BMI between patients and control subjects, we calculated means and standard deviations (SD) and compared the means using an unpaired, two-tailed t-test. Statistical significance was set at α=0.05. The classification accuracy was calculated using a leave-one-out cross-validation procedure with a maximum likelihood classifier as described above.

Results

[Table 1] summarizes baseline characteristics of the study population, including age, gender, BMI and etiology of CS for patients. Patients and control subjects did not differ significantly in terms of age or BMI.

Table 1 Baseline characteristics of the study sample.

	Cushing’s syndrome (CS)		Control subjects
	Males	Females	Males	Females
N (Total)	22	60	32	66
N (Cushing’s disease)	8	37	N/A	N/A
N (Adrenal CS)	8	13
N (Ectopic CS)	3	1
N (Iatrogenic CS)	3	9
Age, years, mean (SD)	53.4 (12.9)	50.2 (7.4)	47.2 (15.0)	46.1 (15.4)
Body mass index, kg/m2, mean (SD)	28.7 (3.7)	30.3 (6.3)	30.9 (5.1)	32.5 (7.4)

[Table 2] summarizes the classification results. The software correctly classified 53.7% of all patients and 70.4% of all controls using frontal- and profile photographs, resulting in a total classification accuracy of 62.8%.

Table 2 Classification results of computer vision software for the detection of Cushing’s syndrome vs. controls showing signs of the metabolic syndrome.

Subjects	Classification accuracy (%)
	N	Females	N	Males
Overall	126	61.1	54	66.7
Control subjects	66	65.2	32	81.3
Cushing’s syndrome (CS)	60	56.7	22	45.5
By Etiology*
Cushing’s disease	37	48.6	8	50
Adrenal CS	13	53.8	8	37.5
Iatrogenic CS	9	88.9	3	66.7
By Course**
Newly diagnosed	26	38.5	14	35.7
Recurrent disease	17	64.7	3	66.7

* Ectopic CS (N=4) not separately calculated; *  Course of disease in N=22 subjects uncertain; * * Course of disease.

Subgroup analyses showed differentiated results. The correct classification rates were higher for females in the CS group, but higher for males in the control group. Classification rates were also higher for subjects with iatrogenic CS than for those with endogenous CS, and higher for those with recurrent disease than for those with newly diagnosed CS, respectively. The highest classification accuracy was achieved for females with iatrogenic CS (88.9%), while males with adrenal CS had the lowest result (37.5%).

Discussion

This study addressed a clinically relevant problem using a novel diagnostic approach and a thorough study design. Computer vision technology is an innovative approach to diagnosing diseases associated with morphological facial changes in an inexpensive and non-invasive manner. We investigated the application of this technology for the differential diagnosis of CS, specifically regarding the advanced problem of distinguishing subjects with CS from subjects with MetS. We found only moderate overall classification accuracy, but highly differentiated results in subgroup analyses.

In our pilot study on the use of computer vision technology for detection of CS, correct classification rates were 85% for patients and 95% for controls using a small sample of female subjects with healthy controls [15]. In the current study we enlarged the sample size, included male and female subjects, and matched the CS-patients with a control group showing signs of MetS. In comparison with our previous results, we found that this method does not perform as well in a less constrained setting. This can be explained by two primary factors that resulted in a rather homogenous sample of study subjects, with only moderate symptoms of CS or clinical features suggestive of it. For one, control subjects were recruited from a sample of patients with clinically suspected CS due to mild suggestive symptoms. On the other hand, patients were recruited at specialized centers for endocrine disorders, which possibly led to comparatively early diagnosis of endogenous CS with only moderate symptoms. Therefore, it is reasonable to suggest that differentiating these two groups is very difficult via an algorithm that is solely based on image analysis and does not incorporate any additional information like medical history or other signs and symptoms. With regard to our previous study on this topic, we believe that comparison with a healthy control group would result in a much better classification accuracy. It is also of note that biochemical testing methods for CS show similar issues with poor performance and false positive results in MetS subjects due to the associated subclinical hypercortisolism [9] [10] [11]. In iatrogenic CS, clinical signs of hypercortisolism are typically accepted as a side effect of immunosuppressive glucocorticoid treatment. Consequently, classification rates were much higher for this group of subjects, which validates the explanation provided for the moderate classification accuracy of endogenous CS.

Several limitations apply to this study. Most importantly, the sample size is comparatively small for training a computer vision algorithm. Training databases used in computer vision typically contain several thousands of images. For example, the Labeled Faces in the Wild (LFW) dataset contains more than 13 000 images of faces and is freely available for computer vision researchers [19] [20]. However, a much larger sample of subjects with active endogenous CS is difficult to recruit due to its low incidence and reversibility of symptoms after successful treatment. In the context of endocrinological orphan diseases (i. e., CS), the sample size of this study is sufficient and comparable to that of phase III studies. For example, a 12-month phase III study of pasireotide in Cushing’s disease included 82 subjects [21]. Though it is currently not feasible, we believe that a larger sample would result in better classification results.

The placement of graph nodes was performed in knowledge of the diagnosis. However, it was done according to a standardized operating procedure with predefined node positions. Previous work from the authors‘ research group has additionally shown that blinded placement of the graph has no significant effect on classification results [16].

Another potential limitation comes from the fact that only images of caucasians subjects were analyzed in this study. The applicability of results to other ethnicities is therefore limited. There was also a gender imbalance with 66 female subjects and 32 male subjects, but this approximately corresponds to the epidemiological gender distribution of Cushing’s syndrome (3–8 females per 1 male) [22] [23].

Computer vision technology has progressed rapidly in the previous years. The study was planned in 2009, and the employed classification algorithm was last updated in 2014. However, re-analysis of the data using a different algorithm would have limited comparability with previous results from our group, and provided uncertain additional benefit regarding the interpretation of study results. We believe that with recent advances in this area, especially 3-dimensional mapping and classification of faces, medical applications of curre nt computer vision algorithms might show better classification accuracy [24].

Computer vision technology is widely used and continuously developed in academic as well as government and private sector research, most importantly for the identification of human faces, but also for classification and mapping of objects and spaces for augmented and virtual reality applications. It is therefore a current research subject with great potential for medical use. It has previously been shown that this technology can be used to detect acromegaly and various genetic syndromes on facial photographs [13]. Hypothetically, a computer vision algorithm could simultaneously analyze a photograph regarding multiple diseases (e. g., acromegaly, CS, Graves‘ disease). Current face detection algorithms can be run locally on smartphones, possibly with medical image analysis performed locally on the device to prompt a medical consultation. Another hypothetical possibility to automatically capture and simultaneously assess morphological changes of the face and entire body associated with CS, such as a round face, central obesity with thin extremities, or dorsal cervical fat pats, is the use of 3-dimensional body scanning [25] [26].

Conclusions

We were able to show that computer vision technology can be used to distinguish CS and MetS on facial photographs with moderate accuracy. The results are promising for further research on medical applications of this technology. We discussed factors that contributed to these results and measures that might improve classification accuracy.

Detection of CS as early as possible is crucial, as ongoing hypercortisolemia may have devastating (and sometimes even lethal) consequences and several of its comorbidities are long-lasting (e. g., proximal myopathy) even after biochemical cure [5]. Computer vision technology could be established to support early detection in a cost-effective and non-invasive way and precede biochemical diagnostic methods. It might be particularly helpful in areas where access to specialized physicians and laboratories is limited. Endocrinological telemedical services appear to be a safe and effective way of delivering outpatient care and could be combined with innovate screening measures. For example, image analysis could be used for screening and followed up by a telemedical consultation with an endocrinologist [27]. The prevalence of CS is suspected to be considerably higher than current estimates, so an easily applicable screening method in primary care might be useful for early diagnosis [28]. Advanced machine learning algorithms that incorporate information from multiple sources (e. g., image analysis, body scanning, and medical history) might be developed to further improve automatic classification accuracy.

However, we believe that image analysis should only be used for screening and differential diagnosis and cannot replace a sufficient diagnostic work-up including personal consultation, physical examination and biochemical testing.

Funding

This work has been partially funded by a German Research Foundation (DFG) grant WU 314/6–2 to R. P. Würtz and by a grant from the Else Kröner-Fresenius Stiftung to M.R. in support of the German Cushing’s Registry CUSTODES (2012_A103 and 2015_A228).

Conflicts of Interest

KHP, RPK, RF, CMB, SZ, MR, MW, RPW, TD, MQ report no conflict of interest. APAK has been reimbursed conference delegate fees by Pfizer, Novartis and Sandoz. GKS received fees for consultancy and/or reimbursement of conference fees and/or travel grants and/or research funding from HRA, Ipsen, Lilly, Novartis, NovoNordisk, Pfizer, Sandoz, and Shire. HJS received fees for consultancy and/or reimbursement of conference fees and/or travel grants and/or research funding from Novartis, Pfizer, Ipsen, Lilly, NovoNordisk, Sandoz, and Shire.

^* Medicover Neuroendokrinologie, Prof. Stalla und Kollegen, Munich, Germany.

^† Institute of General Practice and Family Medicine, Ludwig-Maximilians-Universität, Munich, Germany

^‡ Department of Internal Medicine I, Klinikum Augsburg, Augsburg, Germany

^§ Rheumazentrum Erding, Erding, Germany

^** Zentrum für Endokrinologie und Stoffwechsel Nymphenburg, Munich, Germany

• References

• 1 Nieman LK, Biller BM, Findling JW. et al. The diagnosis of Cushing’s syndrome: An Endocrine Society Clinical Practice Guideline. J Clin Endocrinol Metab 2008; 93: 1526-1540
• 2 Biller BM, Grossman AB, Stewart PM. et al. Treatment of adrenocorticotropin-dependent Cushing’s syndrome: A consensus statement. J Clin Endocrinol Metab 2008; 93: 2454-2462
• 3 Pivonello R, De Leo M, Cozzolino A. et al. The Treatment of Cushing’s Disease. Endocr Rev 2015; 36: 385-486
• 4 Dekkers OM, Biermasz NR, Pereira AM. et al. Mortality in patients treated for Cushing’s disease is increased, compared with patients treated for nonfunctioning pituitary macroadenoma. J Clin Endocrinol Metab 2007; 92: 976-981
• 5 Berr CM, Stieg MR, Deutschbein T. et al. Persistence of myopathy in Cushing’s syndrome: evaluation of the German Cushing’s Registry. Eur J Endocrinol 2017; 176: 737-746
• 6 Psaras T, Milian M, Hattermann V. et al. Demographic factors and the presence of comorbidities do not promote early detection of Cushing’s disease and acromegaly. Exp Clin Endocrinol Diabetes 2011; 119: 21-25
• 7 Valassi E, Santos A, Yaneva M. et al. The European Registry on Cushing’s syndrome: 2-year experience. Baseline demographic and clinical characteristics. Eur J Endocrinol 2011; 165: 383-392
• 8 Alberti KG, Eckel RH, Grundy SM. et al. Harmonizing the metabolic syndrome: A joint interim statement of the International Diabetes Federation Task Force on Epidemiology and Prevention; National Heart, Lung, and Blood Institute; American Heart Association; World Heart Federation; International Atherosclerosis Society; and International Association for the Study of Obesity. Circulation 2009; 120: 1640-1645
• 9 Baid SK, Rubino D, Sinaii N. et al. Specificity of screening tests for Cushing’s syndrome in an overweight and obese population. J Clin Endocrinol Metab 2009; 94: 3857-3864
• 10 Ellis E, Chin PK, Hunt PJ. et al. Is late-night salivary cortisol a better screening test for possible cortisol excess than standard screening tests in obese patients with Type 2 diabetes?. N Z Med J 2012; 125: 47-58
• 11 Srikanthan K, Feyh A, Visweshwar H. et al. Systematic review of metabolic syndrome biomarkers: A panel for early detection, management, and risk stratification in the west virginian population. Int J Med Sci 2016; 13: 25-38
• 12 Schneider HJ, Dimopoulou C, Stalla GK. et al. Discriminatory value of signs and symptoms in Cushing’s syndrome revisited: What has changed in 30 years?. Clin Endocrinol (Oxf) 2013; 78: 153-154
• 13 Kosilek RP, Frohner R, Wurtz RP. et al. Diagnostic use of facial image analysis software in endocrine and genetic disorders: review, current results and future perspectives. Eur J Endocrinol 2015; 173: M39-44
• 14 Schneider HJ, Kosilek RP, Gunther M. et al. A novel approach to the detection of acromegaly: accuracy of diagnosis by automatic face classification. J Clin Endocrinol Metab 2011; 96: 2074-2080
• 15 Kosilek RP, Schopohl J, Grunke M. et al. Automatic face classification of Cushing’s syndrome in women – A novel screening approach. Exp Clin Endocrinol Diabetes 2013; 121: 561-564
• 16 Frohner R. Optimierungsstrategien für Gesichtsklassifikation bei der softwaregestützten Erkennung von Akromegalie [Dissertation]. Ludwig-Maximilians-Universität München 2016
• 17 Frohner R, Würtz R, Kosilek R. et al. Optimierung der Gesichtsklassifikation bei der Erkennung von Akromegalie. Journal für Klinische Endokrinologie und Stoffwechsel-Austrian Journal of Clinical Endocrinology and Metabolism 2013; 6: 20-24
• 18 Günther M, Würtz RP. Face detection and recognition using maximum likelihood classifiers on gabor graphs. Int J Pattern Recognit Artif Intell 2009; 23: 433-461
• 19 Huang GB, Learned-Miller EG. Labeled Faces in the Wild: Updates and New Reporting Procedures. Technical Report UM-CS-2014-003. University of Massachusetts; Amherst, MA, USA: 2014
• 20 Huang GB, Mattar M, Berg T. et al. Labeled Faces in the Wild: A database for studying face recognition in unconstrained environments. Technical Report 07-49. University of Massachusetts; Amherst, MA, USA: 2008
• 21 Colao A, Petersenn S, Newell-Price J. et al. A 12-month phase 3 study of pasireotide in Cushing’s disease. N Engl J Med 2012; 366: 914-924
• 22 Boscaro M, Barzon L, Fallo F. et al. Cushing’s syndrome. Lancet 2001; 357: 783-791
• 23 Newell-Price J, Trainer P, Besser M. et al. The diagnosis and differential diagnosis of Cushing’s syndrome and pseudo-Cushing’s states. Endocr Rev 1998; 19: 647-672
• 24 IEEE. 13th International Conference on Automatic Face & Gesture Recognition (FG 2018). In Xi’an China: Institute of Electrical and Electronics Engineers (IEEE); 2018
• 25 Treleaven P, Wells J. 3D Body Scanning and Healthcare Applications. Computer 2007; 40: 28-34
• 26 Toma AM, Zhurov A, Playle R. et al. Reproducibility of facial soft tissue landmarks on 3D laser-scanned facial images. Orthod Craniofac Res 2009; 12: 33-42
• 27 Wasfy JH, Rao SK, Essien UR. et al. Initial experience with endocrinology e-consults. Endocrine 2017; 55: 640-642
• 28 Guaraldi F, Salvatori R. Cushing syndrome: Maybe not so uncommon of an endocrine disease. J Am Board Fam Med 2012; 25: 199-208