Retinal photograph-based deep learning system for detection of hyperthyroidism: a multicenter, diagnostic study

Thyroid hormones are essential for body growth, neuronal development, and regulation of metabolism [1] Hyperthyroidism is a globally common thyroid dysfunction with potentially devastating health consequences. Hyperthyroidism is a form of thyrotoxicosis due to inappropriately high synthesis and secretion of thyroid hormones [2] In the United States, the prevalence of hyperthyroidism is about 1.2%, with approximately 40% of the patients with clinically apparent symptoms and 60% of the patients having subclinical signs [3] Screening for hyperthyroidism in the general population is not a cost-effective and feasible procedure, because of the relatively low prevalence of the disease and since the examination. Only those who are at high risk due to comorbid conditions, family history, or medication use, are recommended to receive physical examinations and laboratory tests [4] Therefore, hyperthyroidism is frequently unrecognized and untreated. It has been estimated that the prevalence of undiagnosed hyperthyroidism is about 1.72% of the general population Europe [5] It may lead to adverse outcomes and increased costs [6] Improved systems for the detection of hyperthyroidism are thus needed.

Deep learning (DL) is a state-of-the-art technique that allows computational models to learn representations of data with multiple levels of abstraction [7] In the field of medicine and healthcare, DL has been primarily applied to analyze medical images for automatic measurement, augmentation, classification, diagnosis, and even prediction. [8‐11] There is an increasing interest in establishing DL-based low-cost and non-invasive methods trained from color fundus photographs to predict demographic parameters and systematic disorders, including age, gender, cardiovascular risk factors [8], anemia [12] and hepatobiliary diseases [13] Rim and colleagues broadened the applicability of retinal photograph-based DL to predict 47 systemic biomarkers, however, thyroid function has not been well predicted [14] The present study was therefore conducted to develop a DL-based model for the detection of hyperthyroidism based on retinal photographs.

A total of 22,940 retinal photographs from 11,409 subjects were included in the study (Table 1). Among them, 5601 images from 2767 subjects were labeled as hyperthyroidism. We used 19,078 retinal photographs from 9,539 participants for the development of the models, and 3862 retinal photographs from 1870 participants were used as the external validation datasets. The basic characteristics of the study populations were presented in Table 1. In the study population for the assessment of model 1, the median age was 46 (ranging from 20 to 88) and 45 (ranging from 15 to 85) in the development dataset and in the internal validation dataset, respectively. In the study population for the assessment of model 2, the median age was 40 (ranging from 15 to 85) and 41 (ranging from 16 to 88) in the development dataset and in the internal validation dataset, respectively.

In the internal validation dataset, model 1 achieved a higher area under receiver operator curve (AUC) than model 2 (0.907 (95% CI 0.894–0.918) versus 0.850 (95% CI 0.832–0.866)), and model 1 had a higher accuracy than model 2 (0.809 (95% CI 0.790–0.826) versus 0.780 (95% CI 0.758–0.801)) (Table 2). Compared to model 2, model 1 also reached a higher sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and F1 score. Subsequently, model 1 was used for the further external evaluation.

In the external validation datasets, the model reached AUCs ranging between 0.816 (95% CI 0.789–0.846) and 0.849 (95% CI 0.824–0.874) (Fig. 3) and achieved accuracies ranging from 0.735 (95% CI 0.700–0.773) to 0.796 (95% CI 0.765–0.824) (Table 2). Similar values for sensitivity, specificity, PPV, NPV, and F1 score were obtained among all external validation datasets.

In the heatmap visualization, the saliency highlighted the large fundus vessels and optic nerve head areas on the fundus images of patients with hyperthyroidism (Fig. 4). It suggested that the system made the diagnosis based on a fixed pattern.

In this study, more than 20,000 retinal photographs taken using various camera types from 26 centers were used. DL algorithms developed in this study were able to detect hyperthyroidism on retinal photographs with an AUC of ranging between 0.82 and 0.85 and accuracies ranging from 0.74 to 0.80. The key finding shows that a DL-based system can well detect hyperthyroidism from retinal photographs. These results suggest that an automated screening for thyroid diseases may be possible based on routinely taken ocular fundus photographs. The heatmap revealed that the DL algorithm was based on the large retinal blood vessels and on the optic nerve head region.

Previous studies have demonstrated a good performance of artificial intelligence-related and fundus image-based algorithms to differentiate both sexes with an AUC higher than 0.90, and to estimate the age with a coefficient of determination (R²) higher than 0.83 [8, 14] Considering that age and gender might act as confounding factors for the performance of DL system for the detection of hyperthyroidism [23] we trained and validated two models using randomly selected controls (model 1) and age and gender-matched controls (model 2), respectively. The two models both showed high accuracy for the detection of hyperthyroidism, with AUCs higher than 0.85. It suggests that the DL systems could detect hyperthyroidism independently of demographic parameters such as age and sex.

Various efforts have been made to screen and diagnose hyperthyroidism in populations with convenient and low-cost approaches. Sato et al. proposed a novel screening method to assist the diagnosis of Graves’ hyperthyroidism applying Bayesian-type and Self-organizing map-type neural networks which used routine test data including serum concentrations of alkaline phosphatase, creatinine, and total cholesterol [24] The model was trained in 120 women (35 patients with Graves’ hyperthyroidism and 85 healthy volunteers) and then tested in 171 female individuals. It achieved a correct diagnosis in 81–94% for the hyperthyroid patients and in 94% to 98% for the healthy individuals. Similar models were formed base on the examination of 78 male individuals and reached AUCs higher than 0.95 for screening hyperthyroidism in 133 subjects [25] Our study cannot be directly compared to these studies since this is the first investigation to diagnose hyperthyroidism from non-invasive data rather than traditional methods such as biochemical blood examinations and systemic biomarkers.

These findings of the present study illustrate that a fundus photo-based DL system can detect hyperthyroidism. These findings agree with clinical studies showing the involvement of the ocular system in patients with hyperthyroidism. To cite examples, thyroid-associated ophthalmopathy (TAO) occurs mainly in the patients with Graves’ disease. Patients with TAO had an increased choroidal thickness [26] Increased retinal microvascular density was detected in patients with active TAO, while the retinal vessel density in the peripapillary area was decreased in eyes with a dysthyroid optic neuropathy [27] In addition, inactive TAO also showed an altered retinal perfusion as assessed in optical coherence tomography angiography [28, 29] To our knowledge, understanding is still limited regarding retinal change in hyperthyroidism without TAO. Previous study indicated patients with thyroid dysfunction had wider retinal arterioles [30] To better understand the mechanism of the DL-algorithm and to minimize the “black-box effect”, we collected ocular fundus images taken in two general hospitals and four medical examination centers located in North, East, South and West China. We used four different types of fundus cameras (TRC-NW200, CR-2 AF, nonmyd α-DIII, and NT-2000). The DL-algorithm showed reliable and reproducible results in all these six external validation datasets. Although human experts cannot recognize the changes from retinal images in hyperthyroidism, the heatmap visualization revealed that the large retinal vessels and the optic nerve head area were the regions preferentially assessed by the DL-system. This may be explainable since hyperthyroidism is hypermetabolic condition, which leads to the increasement of blood flow rate as well as the dilation of peripheral blood vessels. [31].

The DL system found in the present study may have clinical implications. The use of a computational evaluation of fundus photographs may be promising in screening individuals for hyperthyroidism and other thyroid diseases, so that unnecessary cost may be avoided and social burden is reduced. Future studies may evaluate whether the DL system can be integrated into mobile terminals, such as smart phone apps, to identify hyperthyroidism individuals from populations. Identification of hyperthyroidism at an early stage can assist clinicians to better organize future management strategies and provide more treatment options. Patients undergoing ophthalmic examinations and receiving eye surgeries may benefit from better understanding their general conditions such as thyroid function.

When the results of our study are discussed, its limitation should be taken into account. First, the DL system in this study was trained in participants with overt hyperthyroidism, so that the system was not tested to identify subclinical cases. Second, we failed to develop models to predict the precise levels of serum thyroid hormones, including TSH, T₃, and T₄. Third, we did not include participants with other thyroid diseases, such as hypothyroidism, thyroiditis, and thyroid cancer, due to the relatively low prevalence and few cases. Fourth, the study population consisted mostly of Chinese, so that the results may not directly be transferable on individuals of other ethnicities. Fifth, we collected data mainly from physical examinations, which included a large number of healthy subjects and relatively few hyperthyroidism patients. It caused the imbalance between hyperthyroidism cases and control groups. Sixth, although we established two models, only one model was applied in the external validation, and the performance of the model in external datasets were not perfect (all AUCs < 0.90). Seventh, in this study, we only considered thyroid diseases while other systemic diseases (e.g. hypertension) were not excluded. Although it might add some confounding factors, the results might be more close to real-world application. Eighth, although some confounding factors such as age and sex were matched, other parameters such as diastolic/systolic blood pressure with potential correlations between fundus images were not fully evaluated in this study. The strengths of our study include that it is the first artificial intelligence-based investigation to assess the association between retinal features and hyperthyroidism; that the DL system was developed from data obtained in 26 various data sources in China, with different settings such as hospitals and healthcare center; and that a total of four types of fundus cameras were used in the training and validation of the models, suggesting a wide applicability of the DL system.

Retinal fundus photographs may serve for DL systems for a cost-effective and non-invasive method to detect hyperthyroidism.