Improvised Detection of Diabetic Retinopathy Using Fast R CNN
S. Sridevi 1, Bipin Kumar Sahu 2, Mayank Yadav 3, Badal Kumar 4
1Assistant Professor, Faculty of Computer Science and Engineering, SRM Institute of Science and Technology, Ramapuram, Chennai, India.
2,3,4 Student, Department of Computer Science and Engineering, SRM Institute of Science and Technology, Ramapuram, Chennai, India.
*Corresponding Author: [email protected]
ABSTRACT
Diabetic Retinopathy (DR) is a medical disease that affects the retina in the eyes of people who have diabetes mellitus. It has become a major cause of blindness in diabetes patients as it affects nearly eighty percent of patients afflicted with this disease for longer periods of time. Through this paper, we propose a method for improved detection of diabetic retinopathy by extracting its distinctive features from fundus images and classifying them as DR affected or normal. Firstly, preprocessing of images is done by performing morphological transformations. Then, extraction of features and classification is done using a Fast R-CNN (Fast-Region based convolutional neural network). This paper demonstrates that the efficiency and accuracy of our proposed model using Fast R-CNN is better than other conventional approaches. Our model was able to achieve an AUC score of 0.9298.
Keywords
Diabetic Retinopathy; Fast R-CNN; Fundus image.
Introduction
Humans may be affected by a multitude of eye disorders, and since the eyes are one of the most powerful senses, diagnosing and treating those diseases is critical. Cataract, conjunctivitis, stye, Diabetic Retinopathy etc. are examples of commonly occurring eye illnesses. If such diseases are overlooked for a long time, they will intensify to the point where they cause blindness.
Diabetic Retinopathy is an eye condition that affects diabetics that is caused by elevated blood sugar levels compromising the retinal blood vessels. In the initial stages, DR may show no signs and symptoms but left undiagnosed can eventually cause blindness. DR can take two forms - NPDR i.e.
Nonproliferative diabetic retinopathy and the other being PDR - Proliferative diabetic retinopathy.
NDPR causes microaneurysms in the vessel walls which can sometimes leak fluid and blood into the retina. Microaneurysms are like small, red coloured speckles on the surface of the retina formed by expansion of the blood vessels due to blockage. PDR causes the infected blood vessels to close, causing the emergence of new dysfunctional blood vessels, which contributes to the formation of scar tissue. The retina will separate from the eye as a result of the scar tissue pulling on it.
DR can cause various complications like Vitreous hemorrhage which is the leaking of the blood vessels into the eye, Retinal detachment, Glaucoma and blindness. Since it is an elusive condition, the most recognized way to prevent loss of vision is to visit an ophthalmologist regularly for frequent checkups and keeping diabetes under control.
Figure 1.Various lesion features from fundus image
The methods used to diagnose this problem include comprehensive dilated eye exams, Fluorescein angiography which involves injecting a dye into the bloodstream to capture images of eye to check for leaking and affected blood vessels and Optical coherence tomography (OCT) exams which generates cross-sectional images of the retina for examination of thickness and leakage in retina. A general group of features to be extracted from a retinal image is shown in Figure 1.
There is no cure that exists for DR. So, early detection and preventive treatments are the only ways to manage the progression of this condition. In the NPDR stage, controlling blood sugar levels and managing early symptoms can help to slow down or stop the progression of DR. But if it reaches an advanced stage then the patient might require surgery or laser treatment. Manually diagnosing this disorder is time-consuming and liable to errors. So, the development of methods for Automatic detection and classification of DR is crucial for early diagnosis and for further treatment. The existing methods revolve around examination of fundus images for irregularities manually by a doctor. A sample color fundus image is shown in Figure 2.
The aim of this study is to help with the automated identification of DR by correctly detecting microaneurysms in fundus photographs. For extraction of the features and classification of the image as normal or affected with DR, a fast - region based convolutional neural network (Fast-RCNN) is used.
The purpose of the proposed model is to make the detection of diabetic retinopathy easier and faster with increased accuracy.
Figure 2. Sample Fundus Image
Related Work
Adarsh. P et al. [10] proposed a system for detecting microaneurysms based on multiclass SVM classification. One of the disadvantages of SVM is the time-consuming task of manual data training.
S.S. Rahim et al. [15] used a variety of image processing methods before using the circular Hough transform to find microaneurysms. Following that, decision trees were used to classify the data.
S.S. Rahim et. al [16] proposed an ophthalmic decision support system that acquires, screens and classifies the retinal fundus images automatically, in order to aid in the detection and diagnosis of diabetic retinopathy (DR).
Yitian Zhao et al. [24] used fluorescein angiography (FA) imaging to construct an RVSM (Retino Vascular Structure Map), which helps to demonstrate retinal vessel blockage and leakage. A superpixel-based tool to detect saliencies has been used to recognize retinal leakage regions in FA images.
To distinguish the lesions for DR, Muhammad Sharif et al. [1] used different classification approaches such as multi-kernel KNNs and multi-bagged trees.
Pedro Costa et al. [4] suggested a model called Convolutional BoVW that can learn to extract attributes, encode them, and classify them using classification errors as input.
For detection of red lesions in retinal fundus images, Kedir M. Adal et al. [2] proposed a multi stage automated detection and classification system which classified normalized images using an SVM classifier. It achieved a sensitivity of 80%.
Pedro Costa et al. [5] created a Multiple Instance Learning Framework (MIL) that uses annotations to exploit the information on the picture. Using a novel loss feature that enforces adequate mid and instance-level representations, the Bag of Visual Words (BoVW) improves its decision-making ability.
Xianglong Zeng et al. [25] used Siamese twins like binary convolutional neural networks, which uses two copies of the same CNN, hence the name Siamese Networks. They shared the same set of parameters. The central idea is to use a similarity function, which instead of learning to classify images, tries to show how similar two input images are. They have achieved it by comparing the feature vectors generated by the twin CNNs.
Sehrish Qummar et al. [14] trained five (CNN) models (Dense121, Dense169, Inceptionv3, Resnet50, Xception,) for feature extraction and classification of DR at numerous steps using a public Kaggle dataset containing retina images.
Proposed System
Several morphological operations and segmentation strategies have been proposed in this paper for the identification of microaneurysms that develop during the early stages of DR. The dataset which we have used for this project is Kaggle DR Dataset. During the pre-processing stage, the goal is to make input as balanced and uniform as possible. Hence, various morphological operations are performed, to make the images ready to be fed into the Fast RCNN network. Also, to balance the dataset, augmentation is performed on the minority classes. This includes flipping, rotation, cropping, translation, illumination, scaling etc. Then we pass the images into a Deep ConvNet to give us a feature map. Simultaneously a selective search algorithm generates region proposals. The ROI will then be projected over the feature map and constrained to a distinct dimension by a ROI pooling layer, the output of which will then be fed into the FC (fully connected) layer, which will use the Softmax classifier to perform the classification. The output from the fully connected layer is also connected to the Bbox regressor to generate the bounding boxes for the classified regions. The various advantages are:
1. It can perform competitively on various datasets.
2. It is capable of learning discriminative features.
3. It can drastically reduce the computational load since the image undergoes only one forward propagation per image.
4. Several such optimizations have resulted in improved training and detection time.
The Figure 3 shows the system diagram of the proposed system.
Figure 3. Architecture Diagram of the Proposed System
Methodology Data Preprocessing
Since input images can come in a variety of shapes and sizes, we have to normalize all the input images in a way so that it is convenient for further processing. Hence, several morphological transformations will be performed involving the modification of shape and form of the images. Some of these techniques are mentioned below:
Thresholding: Each of the pixels of the image having a value greater than a certain threshold will be converted to 1, remaining will be 0. This will result in a binary image.
Erosion: This process shrinks lighter regions and enlarges darker regions.
Dilation: This process shrinks darker regions and enlarges the lighter regions.
Whenever dilation is carried out after erosion the process is called opening. It removes small bright spots. Whenever erosion is carried out after dilation, the process is called closing. It removes small dark spots.
Extraction of Features
The photo is fed into the convolution layer, which turns it into a feature map. Simultaneously the imported image is fed into a selective search algorithm which identifies various patterns that form an object which is the micro aneurysms in this case. Based on that it proposes various regions.
Selective search entails creating sub-segments of an input image, then recursively merging the same related regions into larger ones. The filters used in the RCNN's convolution layer isolate similar features from the input images in order to move it on to the next layer. Each of these filters provides a unique function to help in class prediction.
ROI Pooling
After region proposals and feature maps are obtained, the proposed regions are projected onto the feature maps. However, all the ROIs are usually not of the same size. To tackle this problem, quantization is performed wherein a larger varying set of input is constrained into a fixed discrete set of values. This is done because the pooling layer is followed by a fully connected layer, whose fundamental requirement is a fixed input size. The same process is applied to every single RoI of the original image. We might have hundreds or even thousands of 3x3x512 matrices.
Classification
Every one of those matrices generated in the previous step has to be sent through the rest of the network (starting from the FC layer). Unlike a shallow network, where the extracted features are generic, our network extracts individual features. The output layer, the FC layer, is in charge of flattening the inputs it receives from the previous layers. As a result, the outputs are optimal for the network's desired further processing. A Bbox regression layer is also introduced in parallel with the aim of producing bounding box coordinates that can be used to illustrate predicted areas and classes. A uncertainty matrix is created to help in interpreting and visualizing the classification's accuracy.
Results and Discussion
Figure 4. Class Distribution of Dataset
The dataset used has class distribution as shown in Figure 4. Class 0 represents No DR, class 1 represents Mild symptoms, class 2 represents symptoms of Moderate DR. Class 3 represents
proliferated DR and finally class 4 represents severe symptoms of DR. We used 40% of the data set for
predication and the remaining 60% for training our model. As a loss function, we used categorical cross entropy. It can be determined using the following formula:
𝐶𝐶𝐸 𝑝, 𝑡 = − 𝑡𝑜,𝑐log(𝑝𝑜,𝑐)
𝑐
𝑐=1
Where:
c is the no. of classes
t is the target value
p is the predicted value
o represents observation
The training and value loss can be visualized as shown in Figure.5 below
Figure 5. Training and Validation Loss Figure 6. Training and validation accuracy Our model was able to achieve accuracy of about 0.8618 as shown in Figure.6. Finally, the AUC score achieved through this model was 0.9298 as shown in Figure.7 below
Figure.7 AUC curve
The final values of the evaluation metrics are summarized in the table given below:
Table.1 Summary of Evaluation Metrics
Conclusion
This paper describes an improvised method for detecting Diabetic Retinopathy by accurately identifying microaneurysm regions using Fast RCNN. The achieved sensitivity and accuracy outputs further demonstrate that the proposed method is superior for detecting multi stage diabetic retinopathy.
This paper's potential work may include reducing the algorithm's overhead, while enhancing its accuracy.
Metric Value
Training Loss 0.6871 Training accuracy 0.844
Training AUC 0.9268
Validation Loss 0.6256 Validation Accuracy 0.8618 Validation AUC 0.9298
References
[1] Amin J., Sharif M., Yasmin M., Ali H., Fernandes S.L., "A method for the detection and classification of diabetic retinopathy using structural predictors of bright lesions", Journal of Computational Science, vol. 19, pp. 153-164, 2017.
[2] Adal, K.M., van Etten, P.G., Martinez, J.P., Rouwen, K.W., Vermeer, K.A., van Vliet, L.J., "An automated system for the detection and classification of retinal changes due to red lesions in longitudinal fundus images", IEEE Transactions on Biomedical Engineering, vol. 65, no. 6, pp.
1382-1390, June 2018.
[3] Antal B. and Hajdu A., “Improving microaneurysm detection in color fundus images by using context-aware approaches,” Computerized Medical Imaging and Graphics, vol. 37, no. 5–6, pp.
403–408, 2013.
[4] Costa, P., Campilho, A., "Convolutional bag of words for diabetic retinopathy detection from eye fundus images", IPSJ Transactions on Computer Vision and Applications, vol. 9, no. 1, pp.
10, 2017.
[5] Costa, P., Galdran, A., Smailagic, A., Campilho, A. "A Weakly-Supervised Framework for Interpretable Diabetic Retinopathy Detection on Retinal Images," IEEE Access, vol. 6, pp.
18747-18758, 2018.
[6] Datta N. S., Dutta H. S., De M. and Mondal S., “An Effective Approach: Image Quality Enhancement for Microaneurysms Detection of Non-dilated Retinal Fundus Image”, Procedia Technology, vol. 10, pp. 731–737, 2013.
[7] Grinsven M. J. J. P. van, Ginneken B. van, Hoyng C. B., Theelen T. and Sánchez C. I., "Fast Convolutional Neural Network Training Using Selective Data Sampling: Application to Hemorrhage Detection in Color Fundus Images", IEEE Transactions on Medical Imaging, vol.
35, no. 5, pp. 1273-1284, May 2016.
[8] Kumar S. and Kumar B., "Diabetic Retinopathy Detection by Extracting Area and Number of Microaneurysm from Colour Fundus Image", 5th International Conference on Signal Processing and Integrated Networks (SPIN), pp. 359-364, 2018.
[9] Manchalwar M. D. and Warhade K. K., "Histogram of Oriented Gradient based Automatic Detection of Eye Diseases", International Conference on Computing, Communication, Control and Automation (ICCUBEA), pp. 1-5, 2017.
[10] P Adarsh and D Jeyakumari, "A Novel Method for Microaneurysm Detection and Diabetic Retinopathy Diagnosis", IJCA Proceedings on International Conference on Innovations in Intelligent Instrumentation Optimization and Electrical Sciences ICHIOES, no. 6, pp. 42-46, December 2013.
[11] Prasad D. K., Vibha L. and Venugopal K. R., "Early detection of diabetic retinopathy from digital retinal fundus images", IEEE Recent Advances in Intelligent Computational Systems (RAICS), pp. 240-245, 2015.
[12] Qummar, S., Khan, F. G., Shah, S., Khan, A., Ahmad D., Gao, J.,” Deep Learning Techniques for Diabetic Retinopathy Detection”, Current Medical Imaging, Vol. 16, no. 10, pp. 1201-1213, 2020.
[13] Qiao L., Zhu Y. and Zhou H., "Diabetic Retinopathy Detection Using Prognosis of Microaneurysm and Early Diagnosis System for Non-Proliferative Diabetic Retinopathy Based on Deep Learning Algorithms”, IEEE Access, vol. 8, pp. 104292-104302, 2020.
[14] Qummar, S., Khan, F. G., Shah, S., Khan, A., Shamshirband, S., Rehman, Z. U., Khan, I.A., Jadoon, W. (2019). "A Deep Learning Ensemble Approach for Diabetic Retinopathy Detection", IEEE Access, vol. 7, pp. 150530-150539, 2019.
[15] Rahim S. S., Palade V., Shuttleworth J. and Jayne C., "Automatic Screening and Classification of Diabetic Retinopathy Fundus Images", Communications in Computer and Information Science, vol. 459, pp. 113-122, 2014.
[16] Rahim S. S., Jayne C., Palade V. and Shuttleworth J., "Automatic detection of microaneurysms in colour fundus images for diabetic retinopathy screening", Neural Computing and Applications., vol. 27, no. 5, pp. 1149-1164, 2016.
[17] Roychowdhury S., Koozekanani D. D. and Parhi K. K., "Automated detection of neovascularization for proliferative diabetic retinopathy screening", 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 1300-1303, 2016.
[18] Solkar, S. D., Das, L., “Survey on retinal blood vessels segmentation techniques for detection of diabetic retinopathy”, Diabetes, 2017.
[19] Savastano, M. C., Federici, M., Falsini, B., Caporossi, A., Minnella, A. M., “Detecting papillary neovascularization in proliferative diabetic retinopathy using optical coherence tomography angiography”, Acta Ophthalmologica, vol. 96, no.3,pp. 321-323, 2016.
[20] Safi, H., Safi, S., Hafezi-Moghadam, A., Ahmadieh, H. “Early detection of diabetic retinopathy”, Survey of ophthalmology, vol. 63, no.5, pp. 601-608, 2018.
[21] Ting, D. S. W., Cheung, C. Y. L., Lim, G. et al., “Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes”, Jama, vol. 318, no .22, pp. 2211-2223, 2017.
[22] Wang J., Bai Y. and Xia B., "Simultaneous Diagnosis of Severity and Features of Diabetic Retinopathy in Fundus Photography Using Deep Learning", IEEE Journal of Biomedical and Health Informatics, vol. 24, no. 12, pp. 3397-3407, Dec. 2020.
[23] Yu S., Xiao D. and Kanagasingam Y., "Exudate detection for diabetic retinopathy with convolutional neural networks", 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 1744-1747, 2017.
[24] Zhao, Y., Zheng, Y., Liu, Y., Yang, J., Zhao, Y., Chen, D. and Wang, Y., “Intensity and Compactness Enabled Saliency Estimation for Leakage Detection in Diabetic and Malarial Retinopathy”, IEEE Transactions on Medical Imaging, vol. 36, no. 1, pp. 51-63, 2017.
[25] Zeng Xianglong, Chen Haiquan, Luo Yuan and Ye Wenbin, “Automated Diabetic Retinopathy Detection Based on Binocular Siamese-like Convolutional Neural Network”, IEEE Access, vol.
7, pp. 30744-30753, 2019.
