NON-INVASIVE DETECTION OF ANEMIA USING LIP MUCOSA IMAGES TRANSFER LEARNING CONVOLUTIONAL NEURAL NETWORKS

Shekhar Mahmud¹ Mohammed Mansour^2* Turker B. Donmez² Mustafa Kutlu² Chris Freeman³

• ¹Military Technological College, Oman
• ²Sakarya University of Applied Sciences, Türkiye
• ³University of Southampton, United Kingdom

Anemia is a condition in which the number of hemoglobin-containing red blood cells in the blood is reduced. The World Health Organization (WHO) defines anemia as a hemoglobin level in the blood that is less than 13 g/dl in men, 12 g/dl in women, and 11 g/dl in pregnant women [1]. Anemia is most commonly caused by a decrease in red blood cell synthesis or an increase in red blood cell breakdown and loss [2,3,4].Anemia can also be caused by the creation of defective red blood cells in some hereditary blood illnesses.As seen in Figure1, this results in a drop in the average red blood cell count in the blood. Taking intravenous blood from a venous vein and examining it with a hemogram is the gold standard for identifying anemia [5,6,7]. Invasive operations are uncomfortable and difficult to coordinate, especially in pregnant and pediatric patients [8]. The subject must visit a clinic to undergo the necessary procedure. Following the COVID-19 pandemic, executing these operations in medicine, as well as traditional follow-up measures, is no longer possible. Non-invasive anemia monitoring may bring benefits in terms of patient comfort. hemoglobin levels from the color and metadata of smartphone photos shot on the nail bed, and Noriega et al. developed Selfie Anemia, a non-invasive smartphone app that estimates hemoglobin under controlled illumination [9,10,11,12]. Non-invasive methods for anemia detection have been explored, with a focus on the conjunctiva in general [13,14,15]. Patients, on the other hand, will find it incredibly difficult to determine anemia from a conjunctival image using a simple phone camera for IoT connectivity. It has benefits to diagnose anemia non-invasively by taking pictures of the nail bed, hand, and conjunctiva. It has certain restrictions as well. These restrictions include difficulties with accuracy brought on by differences in skin color, the effects of variables, the existence of medical problems, and patient variety. problems exist around data quality, privacy problems, associated expenses, approval procedures, and the requirement for clinical validation. It is crucial to keep in mind that any diagnostic method, including this one, should be used in conjunction with other tests and clinical judgment to achieve accurate findings. Effectively addressing these constraints necessitates study, exhaustive validation methods, and cautious implementation.Images of the lip mucosa can be used to diagnose problems in a special way. Since this procedure is non-invasive, patients won't experience any discomfort or agony as a result of it. It is patient-friendly since it is simple to access and suited for low-cost screening programs. We can eliminate the pain and potential infection risks connected with blood testing by evaluating the lip mucosa. Given that it encourages acceptance and engagement, this strategy is especially appealing to those who are afraid of needles or who work in healthcare environments. Its flexibility for continuous monitoring and use in certain populations, such pediatric patients, emphasizes its promise as a practical and successful diagnostic strategy even more.The effectiveness of lip mucosa analysis in identifying diseases like anemia, however, has to be thoroughly demonstrated via scientific study and clinical trials.Deep learning (DL) and deep transfer learning are important elements of data science, with applications including statistics and predictive modeling [16,17,18,19,20]. Convolutional neural networks (CNN) are specific architectures for input formats such as images, are typically used for image recognition and classification, as shown in Figure 2 [21,22,23,24]. These deep neural networks have proven successful in many real world applications, including: image classification, object detection, segmentation and face detection. Transfer learning takes the classifier layer from a pre-trained CNN and fine-tunes it on the target dataset. This reduces training demands, and is a typical technique for using deep CNNs on small datasets.Machine and deep learning-based disease detection is a significant and revolutionary area of medical study.It has become an essential method for the early and precise identification of a variety of illnesses, including but not limited to cardiovascular diseases, cancer, infectious diseases, and neurodegenerative diseases like Alzheimer's. For example Lei et for Alzheimer's disease diagnosis using complete trimodal images [28]. When analysing massive datasets, such as clinical data, biomarker data, and medical imagery, machine learning makes use of algorithms and computer capacity to find tiny patterns and predictive signals that would escape human observation. By enabling early intervention, optimizing treatment regimens, and cutting healthcare expenditures, this technology holds enormous promise for bettering patient outcomes. Additionally, as healthcare technology develops, machine learning's role in illness detection is expected to expand, bringing with it fresh ideas for providing people all over the world with healthcare that is more accurate, individualized, and accessible. This research is the first to use lip mucous images for anemia detection in the literature. The goal of this study is to predict anemia using lip mucous images which has thin skin tissue, and to confirm the feasibility of detecting anemia in a non-invasive way. This is accomplished by employing CNN transfer learning.Well known CNN models; Xception, MobileNetV2, VGG16 and ResNet50 are used for classification of dataset which contain two lip image classes; anemia and healthy. The classification models, were evaluated using metrics; accuracy, recession, recall, F1 score. The work is of importance because it makes use of AI through DL, which is frequently employed for medical diagnosis and prediction [29,30,31,32]. This study also provides a non-invasive technique that is more practical and available to patients than blood testing, particularly in places where these tests are not easily accessible. Early diagnosis of anemia is essential since it can stop major health effects including fatigue, weakness, and weakened immune system.Additionally, creating and training ML algorithms to interpret photos of lip mucous is a low-cost screening The strategy described in our work shows potential for significantly increasing the accuracy and sensitivity of anemia diagnosis when compared to prior studies and standard diagnostic methods.This growth may be attributed to several factors, including the application of DL algorithms with features selection, the inclusion of various data types and demographic data, a huge and diverse dataset, and careful evaluation using performance measurements. Additionally, the non-invasiveness, cost, accessibility, and capabilities of lip mucosa analysis should lead to better patient compliance and quicker anemia detection. Even while these are crucial steps in demonstrating its superiority in actual clinical practice, thorough validation, comparisons with other diagnostic procedures, and clinical trials are insufficient on their own to substantiate these claims. This article is organized as follows: Section 2 is a discussion of non-invasive approaches for detecting anemia and CNN approaches for anemia detection as well. Section 3 explains the methodology; system design, image processing and CNN models. The results are given in Section 4. Discussion is presented in Section 5. Conclusion are presented in the final Section, as well as avenues for further research.
This section reviews previously used non-invasive anemia detection methods based on conjuctiva and finger tip analyses, and DL models.Lena et al. used DL techniques, specifically CNN, to carry out conjunctival image-based anemia detection [33]. With the use of the palpebral conjunctiva image and the CNN approach, it was possible to discern between normal and anemic situations with better precision. 2000 photographs of the palpebral conjunctiva, which included anemia and normal circumstances, were used in the investigation. The dataset was then separated into 400 photos for model testing, 160 images for validation, and 1,440 images for training. The study's best accuracy was 94 percent, with average values for precision, recall, and F score of 0.94, 0.94, and 0.93 correspondingly.Zheng et al. proposed a novel computational framework that could fully utilize the conjunctiva image's data [34]. They suggested fully exploring both the global and local information included in the image, and to combine the data from these two elements, they selected a two-branch neural network architecture.Zhao et al. developed and validated a DL algorithm to predict Hgb values and screen for anemia using ultra-wide-field (UWF) fundus conjunctiva image images, at the Peking Union Medical College Hospital, their study was carried out [35]. The dataset was constructed using Optos color photographs that were inspected between January 2017 and June 2021. It was created to use UWF images, ASModel UWF.Its performance was assessed using mean absolute error (MAE) and area under the receiver operating characteristics curve (AUC). To create a visual representation of the model, saliency maps were created.The prediction task's MAE was 0.83 g/dl (95 percent confidence interval: 0.81-0.85 g/dl), and the screening task's AUC was 0.93 for ASModel UWF (95 percent CI: 0.92-0.95). The model tended to concentrate on parts of the retina that were missed by non-UWF imaging, such as the regions around the optic disc and retinal arteries.Palacio et al. used YOLO v5 to create a smartphone app for anemia screening [36]. They made advantage of an conjunctiva image collection received from the Universidad Peruana Cayetano Heredia, which features images of young children and their blood test prognosis. Although YOLO v5 performs well when used on a computer, its performance is diminished when used in a mobile application. Despite this, the app's ability to identify anemia has a sensitivity of 0.71 and a specificity of 0.89.
accurately count blood cells from capillaroscopic movies [37]. CycleTrack is designed to mimic the Mitra et al. proposed a unique non-invasive algorithm that identify anemia using human nails [39]. They employ computer vision, ML, and DL concepts in their methodology, and only on the basis of those data they predicted the degree of anemia for participant. They suggest a method that is fully real-time, and this system was able to produce results. Golap et al. created a model for estimate of a non-invasive hemoglobin and glucose level using photoplethysmogram (PPG) characteristic characteristics retrieved from fingertip video recorded by a smartphone based on multigene genetic programming (MGGP) [40].The PPG signal was created by processing the videos. A total of 46 features have been recovered by analyzing the PPG signal, its first and second derivative, and using Fourier analysis. The best characteristics were then chosen using a genetic algorithm and a correlation-based feature selection approach. Finally, a symbolic regression model based on the MGGP was created to estimate glucose and hemoglobin levels. Several traditional regression models were also created utilizing the identical input condition as the MGGP model in order to compare the performance of the MGGP model. By calculating various error measurement indices, a comparison between MGGP-based models and conventional regression models was conducted. Selected characteristics and symbolic regression based on MGGP were used to find the best results (0.304 for hemoglobin and 0.324 for glucose) among these regression models.Haque et al. suggested a unique non-invasive method based on PPG signal to assess blood hemoglobin, glucose, and creatinine levels (DNN) [41]. Using a smartphone, 93 individuals' fingertip videos were gathered. From each movie, the PPG signal is created, and 46 distinguishing features are then taken from the PPG signal, its first and second derivatives, and Fourier analysis. Age and gender are also taken into consideration because of their significant impacts on hemoglobin, glucose, and creatinine. To reduce redundancy and over-fitting, the best features have been chosen using genetic algorithms (GA) and correlation-based feature selection (CFS). Finally, using the chosen characteristics, DNN-based models were created to predict the blood levels of hemoglobin, glucose, and creatinine. The method offers R 2 = 0.922 for Hb, R 2 = 0.902 for Gl, and R 2 = 0.969 for Cr as the best-estimated accuracy. [42]. The developed frequencydomain multidistance approach (FDMD), based on a non-contact oximeter, provided data on total Most of previous methods detected anemia using data as conjunctival images, human nails and fingertip. ML, DL and CNN were used in some of them to develop invasive and noninvasive methods for detecting anemia. The performance still need to be improved through applying and using new data and simple methods. This study importance as it is the first to deal with lip mucous images for predicting anemia using CNN. The study's use of DL, which is frequently employed in the medical field for prediction and diagnosis, is what provides it its significance [29,30,31,32].

The classification problem is to detect anemia using a collected lip image dataset. The study starts by building the dataset which has two lip images types; healthy and anemia. Following this, digital images processing was performed, CNN transfer learning models for classification then were used and then evaluated to establish the best model. Figure 3 shows the flow chart of the study. The convenience sampling approach was used to determine the study group [44]. Table 1 The experimental setup shown in Figure 4 was designed to measure the facial features of the participants, and it is designed to display only the lip area with the help of an adjustable frame. A camera captured a high resolution image (4896x2752) of each participants. Data collection was carried out by a team of experienced medical doctors.
To process and analyze data, a custom Python application was created. Firstly, the participant's lip contour was determined using corner detection, threshold value, and framing. The framed digital image was then converted to rgb formats as shown in Figure 5, after which it was analysed and classified using CNN structures (see Figure 6).DL require large amounts of data, however, data is not always available in all cases [45]. Therefore, the data extension was applied to improve data diversity by making minor modifications to existing data copies or by creating synthetic data from existing data. Some of the techniques used for data expansion are; rotation, flipping, shear, brightness or contrast changing, cropping, scaling and saturation. In this research, horizontal and vertical flipping, and rotation were applied to dataset. A total of 1380 images were developed using data augmentation techniques. For CNN algorithms to be practical and successful in addressing real-world issues, they must perform well [46,47]. High-performing algorithms may produce more precise predictions, analyze data more quickly, scale to handle big datasets, and be easier to understand, which improves outcomes and decision-making.In ML, classification models are frequently used to forecast outcomes based on a set of traits. There are a number of widely used measures available to assess the performance of such models. The problem's nature, class balance, and the model's intended result all influence the measure that is used [48,49,50].A straightforward statistic called accuracy counts how many of the model's predictions were accurate [51]. When there is a class imbalance in the data, it might not be the best option. When minimizing false positives, precision indicates the percentage of true positives across all positive predictions provided by the model [52,53]. Contrarily, recall assesses the proportion of real positives among all of the actual positive instances in the data and is helpful when the objective is to reduce false negatives [53,54]. F1 score is the harmonic mean of precision and recall and is useful to balance the importance of both [46]. Finally, byshowing the quantity of true positives, true negatives, false positives, and false negatives for the specified model, the confusion matrix offers a more thorough understanding of the performance of the model than any single statistic [55,56].A DL model's performance evaluation is crucial to determining its efficacy and pinpointing areas for development. The model should be evaluated using a variety of metrics, and while selecting a statistic, the problem's context should be taken into account.
The CNN architecture was applied to detect anemia in lip mucous images. Experiments were performed in the Python programming language, as well as its libraries Keras, sci-kit and Tensorflow. Before the classification process, the images were prepared and augmented in order to improve the accuracy of the model and reduce the degree of model overfitting.The number of epochs, hidden layers, hidden nodes, activation functions, dropout, learning rates, and batch size are used to fine-tune the model. Hyper parameter tweaking has an impact on the performance of the model. hyper parameter used are shown in during the network's training phase. To maximize the targeted performance measurement, the learning rate was tested at various settings. The total number of images in the collection was used as the basis for the validation procedure. The accuracy improved as different epochs and batch sizes were adjusted.The effectiveness of a model was assessed and validated using training, testing, and validation techniques.The training and validation accuracy as well as the loss of anemia detection for the best model's results are shown in Figure 7 As seen in Table 3, the confusion matrices shows the predicted classes and their disturbance.Table 3 shows the confusion matrix after it has been calculated. All classes' model performance is precisely measured. The accuracy is described in the following manner. Accuracy = T P + T N T P + T N + F P + T N(1)
In this study, CNN model were used to detect anemia using lip mucous images. The findings of this study are consistent with other studies that used DL models for invasive and noninvasive methods for predicting anemia. The performance of CNN models in predicting anemia was evaluated using accuracy, specificity, precision, recall, F-measure. The accuracy performance of the studied models were high and significant.Xception reported the best accuracy, sensitivity, and F1 measures for anemia prediction.Limited methods were found in the literature to detect anemia. When comparing the test results obtained from this application with laboratory data, the K-means clustering approach, for instance, revealed an accuracy of 90% when utilized to carry out conjunctival pallor image-based anemia identification [57].In order to estimate HGB levels non-invasively, ANN were used to evaluate photos of fingertip obtained with a smartphone camera. The model's hemoglobin levels and the gold standard hemoglobin levels were found to be correlated with 0.93 [58]. CNN was techniques was used to carry out conjunctival image-based anemia detection with accruacy of 94 % [33]. YOLO v5 was used to detect anemia using conjunctiva image collection with sensitivity of 0.71 and a specificity of 0.89 [36]. AlexNet was used to calculate total hemoglobin concentration by developing frequency-domain multidistance approach, based on a non-contact oximeter, provided data on total hemoglobin with accuracy of 87.50 % [42]. Compared to these methods that used CNN and DL in general for anemia detection, the current study is a noninvasive method uses CNN models to detect anemia with higher accuracy; 99.28 % using xception (see Figure 10). This new method is simple and can be developed for real time anemia detection.Lip pallor, which is characterized by the pallor or loss of natural color in the lips, has substantial advantages as a non-invasive diagnostic site when compared to other human body parts. First off, because the lips are so visible and accessible, there is no need for specialized equipment or invasive treatments to inspect them. Lip pallor is a practical alternative for diagnostic methods because to its accessibility, allowing rapid and little intrusive patient evaluations.On the lips, there is also a dense vascular network with many tiny blood vessels near to the surface. Variations in blood flow and oxygenation, which typically manifest as changes in lip color, may be immediately seen because to the vascular richness. Because of the close association between lip color and the circulatory system, the lips are ideal for diagnosing blood-related illnesses including anemia. This gives current health-related information about a patient.Because lip pallor analysis is non-invasive, patient comfort and compliance are improved. Whether it comprises basic eye exams or more advanced imaging methods, the procedure is safe and suitable for people of all ages. Due to the fact that it is typically socially acceptable in all cultures to examine one's lips, this technique also complies with moral principles and cultural norms. A patient may be more eager to take part in lip-based diagnostic procedures if they feel accepted.Because of developments in imaging technology, it is now feasible to do exact color and texture analyses of the lips in high-resolution pictures.This level of specificity ensures that lip pallor analysis will always be a useful and economical screening approach in medical settings, making it an important tool in the field of non-invasive diagnostics. It is necessary for spotting minute indications of disorders like anemia.Given that DL is still being used in medical research in its infancy, this finding also makes an important addition to that field. This study may serve as a starting point for future work on creating ML-based tools for anemia detection and diagnosis, which might result in even more precise and potent diagnostic equipment.Our findings suggest that deploying CNN techniques for anemia detecting the will help in diagnosing using classification, which will in turn aid in the development of effective preventive interventions. Thus, this research not only addresses the integration of innovative technology for the prediction and diagnosis of low hemoglobin level but support the medical system, it assesses the prediction power of several CNN algorithms.