Artificial Intelligence-Based Eyefeature Imaging for Early Diabetes Aided Diagnostics
Citation:Kai Jiang (2020) Artificial Intelligence-Based Eyefeature Imaging for Early Diabetes Aided Diagnostics. J Biomed Eng 1: 1-10.
Diabetes is one of the most common metabolic diseases in the world [1]. In the past several decades, the morbidity of diabetes has continually increased. According to the Sixth Diabetes Epidemiological Investigation in China, the morbidity of diabetes in 1980 was 0.67% and increased to 9.65% in 2010 [2]. The most widely used method for clinical diabetes diagnosis is blood glucose level detection. However, frequent blood glucose testing causes continual harm to diabetics, which cannot meet the needs of early diagnosis and long-term tracking of diabetes [3-5]. Thus non-invasive adjuvant diagnosis methods are urgently needed, enabling early screening of the population for diabetes, the evaluation of diabetes risk, and assessment of therapeutic effects.At present, fund us imaging is the most common method for image diagnosis of the eye, and it is one of the most common methods to screen for diabetic retinopathy (DR) in the clinic. Fund us imaging can show the image of the retina objectively [6]. Currently, the common methods for fundus imaging are a fundus camera, fundus fluoresce in angiography, and retinal tomography, though the fundus camera is the most commonly used clinically [7-8]. However, fundus imaging-based diabetes diagnosis requires a complex and expensive system for fundus detection and is easy confused by eye diseases, making it unfit for early diabetes diagnostics. Further, it only processes images of the retina, ignoring relevant information on the sclera.
Eye feature imaging is a very important diagnostic technique in traditional Chinese medicine (TCM). Two thousand years ago, "The Inner Canon of Huangdi" recorded a method that diagnosed diseases by observing eye features [9]. Copious clinical experiences indicate a close relationship between eye features and visceral organ changes. When visceral organs appear abnormal or undegopathological changes, eye features present corresponding phenomena. For instance, Zhu [10] found that according to the abnormal signal above eye, e.g., eyelid xanthelasma, they can estimate the severity and stage. Song [11] analyzed eye feature images of 150 peptic ulcer patients. He reports that there were abnormal blood vessels in the digestion area of the sclera of these patients. These vessels changed indirection, curvature, and color.
Peng [12] summarizes the basic theory and main methods of eye diagnosis. Aside from TCM, he also references iris diagnostics and fundus image analysis using modern diagnostic equipment. However, eye feature imaging-based TCM diagnostic techniques mainly rely on manual methods and lack automated detection and analysis instruments. In addition, the comprehensive effect of analyzing multiple eye features is more accurate than a single feature, but this analysis relies heavily on physician experience and currently lacks uniform standards and methods that are easy to generalize. These are serious restrictions to the application of eye feature diagnostic techniques. In recent years, with the rapid development of artificial intelligence (AI), the application of AI in medicine has become popular. Medical imaging is a very common and useful tool for disease detection, but medical imaging often has a large scale and can be disordered. Thus, analysis by AI is a fast and effective alternative. Kaushal [13] developed an Eye Art system for fully automated screening of diabetic retinopathy patients, yielded results with good sensitivity and specificity. Piyush Samant [14] analyzed 338 subjects, including 180 diabetic and 158 non-diabetic individuals. He obtained infrared images of these patients, used image segmentation and feature extraction to process the images, and ultimately found that the classification accuracy of the diabetic and non-diabetic groups was 89.63%. Varun Gulshan [15] processed a huge data set (11,711 fundus images from 5871 patients) by deep learning for detection of diabetic retinopathy. The results for EyePACS-1 (it's a data set consisted of 9963 images from 4997 patients) had a sensitivity of 97.5% and specificity of 93.4%. Moving forward, it will be very important to incorporate AI to improve eye feature imaging for TCM diagnostic techniques and systems. In this paper, a novel AI eye feature imaging technology and system were developed. Eye feature images of 127 diabetics and 50 general controls were analyzed, and an accuracy of 84.9% for diabetes diagnosis was obtained from the sclera images of these patients. Additionally, a statistical association was established between five eye features ("Yellow Area", "Gray Speck", "Spot", "Hillock", and "Moon Halo") and three typical clinical testing targets. Statistical analysis revealed that with increasing triglyceride levels, "Yellow Area" features intensified in both size and color. With the proposed method, non-invasive diabetes adjuvant diagnostics and complications risk assessment can be achieved, offering a potential approach for early diagnosis and long-term diabetes monitoring.
Due to its multilayer quasi-sphere structure, the imaging shadow from the illumination source is challenge for eye feature imaging. To solve this problem, a slit lamp microscope was invented for ophthalmological diagnosis [16] , with which doctors can observe flaws in the eyeball in a narrow zone using a slit light produced by the slit illumination source, free from the illumination source's reflection shadows. However, it requires an additional scan of slit light to acquire an overall image of the eyeball. Moreover, it is rather time-consuming to splice images taken with a slit light scan. Here, an AI eye feature imaging approach was proposed to avoid the illumination source's reflection shadow disturbance (Figure. 1). To eliminate the interference of the illumination source's reflection shadows on eye feature imaging, an AI eye feature imaging system was developed as shown in Figure 1(b), where S is the illumination source of a 1-W white light LED, M1 is the cross guiding light of a 1-W green LED, the Lens has a100-mm focal length, CCD is a Canon 5D, S1 is one of the reflection shadows of the illumination source S, Ψ is the angle between the optical axis and S, and ø is the angle between the optical axis and the pupil. To build the neural network-based black eye auto-tracking and white eye auto-focusing method, the optimum values of Ψ and ø were obtained at ~40-50°, and~65-80°, respectively, when all of the illumination source's reflection shadows were focused into a small point and superimposed onto the pupil. Then, the white eye can be imaged clearly without any interference from the illumination source's reflection shadows as shown in Figure 1(d) up.
Similarly, to rotate the eyeball downward, left, right, auto-focus, and photograph the white eye synchronously, images of the entire white of the eye without the illumination source's reflection shadows can be obtained as shown in Figure 1(d) down, left, and right. The abovementioned processes can be finished in 3 min. An additional description of the AI eye feature imaging is shown as Figure 2.
AI-based eye feature imaging for health analysis Figure 2 shows the main processing methods of the AI eye feature imaging system for health analysis.
First, the shadow less eye feature imaging in four directions is performed as in Figure 1. For different people's eyes, the optimum values of Ψ and ø obtained will vary, the cross guiding light M1 is used to direct the user to rotates his eyeball and adjusts his pupil position, and makes negative feedback for the black eye auto-tracking, until the corresponding illumination source's reflection shadows converged at one point and are superimposed into the pupil. After achieving neural network-based black eye auto-tracking, the neural network-based white eye auto-focusing and photographing is sequentially finished in ~10 s. The entire set of white eye images free from the illumination source's reflection shadows are obtained within 3 min, including eight original eye images of the left eye and the right eye, where all eye whites are clear and shadow less.
Second, eye features are extracted and identified by a 30-layer deep convolutional neural network. By marking five eye features, including "Yellow Area", "Gray Speck", "Spot", "Hillock", and "Moon Halo", we trained this neural network so that it can recognize the five eye features on the spherical image obtained in the previous step. In particular, due to the high cost of eye image collection, the samples used for training were limited, so we used a small sample learning algorithm to extend the training data.
Third, to analyze the correlation between the five eye features and diabetes, we established a four-layer neural network, whose input layer is the number of five eye features for each spherical image sample, and the output layer is negative or positive for diabetes. Thus far, an integrated, full-featured, eye feature-assisted diabetes diagnostic neural network system was established through the above three steps.
Finally, eye feature-based health analysis and diagnostics are performed. Using this neural network system, the subjects were photographed, and the pictures were corrected, identified, and analyzed within a few minutes. Therefore, an initial diagnosis of diabetes can be obtained. Whether the subject has diabetes or the risk of diabetes in the future will be forecasted for early warning.
Based on above AI eye feature imaging and system developed in Figure 1(c), white eye images free from the illumination source's reflection shadows are obtained within 3 min using the following steps: First, the eye must be placed close to the location window of the eye frame, and the upper and lower eyelids are carefully open with both hands. Then, the eyeball is rotated up, and the pupil position is synchronously adjusted under the guidance of the cross guiding light M1.Third, the optimum values of Ψ and ø are obtained, and the neural network-based white eye auto-focusing and photographing are sequentially finished. Finally, the eyeball is rotated down, left, and right, the pupil position is synchronously adjusted, the camera auto-focuses, and the photograph is captured.
Eight original eye images of the left eye and the right eye where all white eyes are clear and shadow less were obtained as shown in Figure 3. In the experimental section, the eight original eye images were divided into the inside of the left, the outside of the left, the downside of the left, the upside of the left and the inside of the right, the outside of the right, the downside of the right, and the upside of the right for both eyes of each subject, which belong to four regions of A, B, C, and D for statistical analysis of eye features and health.
With the AI eye feature imaging and system that we developed, 1416 original eye images of the left eye and the right eye where all of the white of the eye was clear and shadow less were obtained from 177 subjects (include 127 diabetics and 50 general controls).By extracting eye features with a 30-layer deep convolutional neural network, these original eye images were analyzed and yielded the following results. One hundred twenty-six of the 177 subjects had the "Yellow Whole eye partial enlargement Area" feature as shown in Figure 4(a), 14 had the "Gray Speck" feature as shown in Figure 4(b), 50 had a "Spot" feature as in Figure 4(c), 73 had the "Hillock" feature as in Figure 4(d), and 11 displayed the "Moon Halo" feature as shown in Figure 4(e) Of these, more than 96% of diabetic patients have one or more eye features. The statistical data from the 127 diabetics and 50 controls are shown separately in Tables 1 and 2, respectively, from the detailed eye feature classification data (see attachment data all. xlsx; data_whole people sheet). Extraction Feature marked.
Then, 70% of the data including, 89 diabetes patients and 35 general controls were randomly chosen for training. The data of diabetes clinical diagnosis from blood glucose level detection and eye features of the same clinical samples were used to train the 4-layer diabetes diagnosis neural network as shown in Eqs. 1 and 2. The details data of the124samples for training are shown as attachment data_all.xlsx in sheet data AI train.
The 4-layer diabetes diagnosis neural network has 20 input nodes, two output nodes, two hidden layers, and 10nodes in each hidden layer. The activation value of the i node of the j layer is: 
The activation values of the nodes on the j layer are:
In Eq. 2, is the number of nodes in the j layer, n is the number of nodes in the j-1 layer, is the weight α,is the activation value, and j is the bias. Based on above machine learning of diabetes diagnosis neural network, eye feature-based health analysis and diagnostics were performed on another 38 diabetics and 15 controls, including forecasting and early warning, the aide diagnostics of diabetes, etc. One thousand sixty eye features from another 38 clinical diabetics and 15 control were used to test the coincidence rate of the eye feature-based AI analysis system, and the AI system diagnostic results are presented in Table 3.
Compared to the clinical diagnoses based on blood glucose level detection, the accuracy of diabetes diagnosis obtained was 84.9% from the eye feature-assisted diabetes diagnostic neural network system. In fact, most of samples whose two diagnostic results did not correspond with the AI eye feature diagnosis system considered them to be positive for diabetes, while the clinical diagnosis was negative. And from their biochemical analysis data, although they did not meet the clinical diagnostic criteria for diabetes, many people had already high blood sugar. In this case, we believe that the subject has a higher risk of developing diabetes, and we should warn them. The details are shown as data_all.xlsx in sheet data AI result.
Triglycerides (TRIG) are the most common lipid in the human body. The ideal level of TRIG should be < 1.70 mmol/L. An increasing concentration of TRIG in serum is an important index of coronary heart diseases, especially for diabetics. For diabetics, when the TRIG concentration is > 2.26 mmol/L, diabetic complications are likely to happen. An important factor of arteriosclerosis is Low Density Lipoprotein (LDL), whose normal concentration should be < 3.12 mmol/L. Total cholesterol (TCHO) is a synthesis of all of the lipids in blood. If one's concentration is > 5.72 mmol/L, he/she will be diagnosed with hyperlipidemia. Considering the most common eye feature of "Yellow Area" in 113 diabetics, which is 89% of the 127 diabetics, the eye feature of "Yellow Area" was used to further analyze the development of diabetes. Subjects without "Yellow Area" features were defined as Normal type, those with "Yellow Area" features in merely one or two parts were defined as Moderate type, and those with "Yellow Area" features in three or four parts were defined as Serious type. There were 15 Normal type patients and 28 Serious type patients among the 113 diabetics. A statistical association was established between the eye features and three typically clinical biochemical testing targets of TRIG, LDL, and TCHO as shown in Table 4.
As shown in Table 4, the levels of the three biochemical indices in Serious type patients were obviously higher than those in the Normal and Moderate type. Further, the levels in Normal type patients were lower than the normal values. With "Yellow Area" appearing in more parts, the level of TCHO also increased, and more patients showed a dangerous level of TCHO, which was > 6.00 mmol/L. Table 5 shows the proportion of patients who had a dangerous level of TCHO in the Normal, Moderate, and Serious groups. In Serious type patients, the proportion reached 60.7%, indicating that these patients had more risks of hyperlipidemia.
The number of "Yellow Areas" appearing in different parts of the eye, the sizes, and the color depths of "Yellow Areas" also displayed obvious relativities. For patients whose TCHO levelswere8.82 mmol/L, which is a dangerous signal of hyperlipidemia, the sizes (~4512pix) of the "Yellow Areas" were almost fourfold larger than that (~1052pix) in the patient whose TCHO levelswere4.66 mmol/L (which is a normal level) as shown in Figure 5. To assess color depth, HSB channels (Hues, Saturation, and Brightness) were used. The color depth of "Yellow Areas" in A(S=50% in the HSB channel) were also deeper than in B(S=40% in the HSB channel), which also verified that the severity level of diabetes from the color depth and size of the "Yellow Area" has an obvious positive correlation with the blood lipid level. The above results indicate that with the proposed technology of the AI eye feature imaging and analysis method, severity levels of diabetes can be conveniently judged. Further, a positive correlation was also revealed between the eye features and the levels of TCHO as determined by clinical biochemical testing.
Eye feature diagnosis is a very important diagnostic technique in traditional Chinese medicine, which has a >2000 year long history and comes from the accumulation of a large number of folk clinical practice experiences. With recent academic developments, it has also been proven that eye features have a potential connection with some diseases. In this paper, a novel technology for AI eye feature imaging was introduced, making eye feature diagnosis more accurate and automatic. Using this system, eight original shadow less white eye images of each eye for each person could be obtained within 3 min, which offered reliable eye feature data free from the illumination source's reflection shadows for subsequent analysis.
With the developed AI system, eye feature images of 127 diabetics were analyzed, and the accuracy of diabetes diagnosis obtained was >84.9% from the eye feature images of patients showing "Yellow Area", "Gray Speck", "Spot", "Hillock", and "Moon Halo" in sclera. Some general control, despite not clinically reaching diabetes, already have obvious relevant eye features, and we believe that they are at higher risk for diabetes. The amount of data we used to train the AI system was limited, so a small sample learning method was used to optimize the algorithm. Larger sample sizes and more advanced small sample learning methods may increase the accuracy of the diagnosis.
By analyzing 1016 original eye images from127 patients, the "Yellow Area" was found to be one of the most common features associated with diabetes. In these diabetic patients, the severity level of diabetes from the eye feature of "Yellow Area" was also positively correlated with the level of TRIG, LDL, and TCHO. This study indicates that from the eye features of the sclera, the complications of diabetes can be conveniently and rapidly analyzed. The larger the size and deeper the color of the "Yellow Area", the greater the possibility that patients will suffer from cardiovascular diseases.
With this AI eye feature imaging and analysis method, diabetic patients' health conditions can be rapidly, noninvasively, and accurately analyzed, which offers a platform for noninvasive forecasting, early diagnosis, and long-term monitoring for diabetes and its complications.
In addition, we are trying to use more similar methods to diagnose more diseases, such as the polycystic ovary syndrome being studied, and we have found that the corresponding main eye feature is the vascular network in the white eye, which is quite different from diabetes.
This work was supported by the National Natural Science Foundation of China (61927819, 81827808, 81327005), the National Key R&D Program of China (2018YFA0704004), the Beijing Municipal Natural Science Foundation (4142025), the Beijing Lab Foundation (BJLAB-2019), and the Tsinghua Autonomous Research Foundation (2018Z05JZY013).
No competing financial interests exist.
