A Novel Approach for the Early Detection of Rheumatoid Arthritis on Hand and Wrist Using Convolutional Reinforcement Learning Techniques
Krishnachalitha K C1, Dr.C.Priya2
1 Research Scholar, Department of Computer Science,Vels Institute of Science, Technology and Advanced Studies (VISTAS), Chennai.
2Research Supervisor and Associate Professor,Department of information Technology,School of Computing Sciences,Vels Institute of Science, Technology and Advanced Studies (VISTAS), Chennai
ABSTRACT
A reliable provocative issue which impacts the joints by harming the body's tissue is called as Rheumatoid arthritis . As needs be, the ID and ID of rheumatoid arthritis by hand, particularly during its unanticipated turn of events or pre-expressive stages, requires an extraordinary construction analysis. The standard end technique for Rheumatoid Arthritis (RA) recalls for the assessment of hands and feet radiographs. Notwithstanding, still for clinical experts it winds up being an unconventional endeavor considering the way that regularly the right completion of the disease relies on the exposure of unfathomably subtle changes in the typical eye. In this work, we built up a design using Convolutional Neural Networks (CNN) and Reinforcement Learning Technique for detecting RA from hand and wrist MRI. For this we took 564 cases(real information) which provided a precision of 100 %. Compared to the existing system, the system showed a high performance with very good results. This model is highly recommended to detect Rheumatoid arthritis automatically ,without human intervention.
Keywords: Machine learning, convolutional neural networks, deep learning, rheumatology,rheumatoid arthritis, artificial intelligence
Introduction
In around 1 % of the world population [55], Rheumatoid arthritis is an unobtrusively common disease, Rheumatoid arthritis is a persistent incendiary problem which will influence very your joints. In certain individuals, the skin, eyes, lungs, heart and veins can be affected. Rheumatoid arthritis is a autoimmune disease. Rheumatoid arthritis happens when your framework erroneously assaults your own body's tissues. Rheumatoid arthritis influences the liner of your joints, causing a difficult expanding which will in the long run end in bone disintegration and joint disfigurement. The aggravation identified with rheumatoid arthritis is the thing that can harm different pieces of the body additionally . While new kinds of prescriptions have improved treatment alternatives significantly, extreme rheumatoid arthritis can in any case cause actual handicaps.
Fundamentals that may develop your risk of rheumatoid arthritis include:
Your sex- Ladies are almost certain than men to create rheumatoid arthritis.
Age- Rheumatoid arthritis can happen at whatever stage in life, on the other hand it usually starts in middle age.
Family ancestry- On the off chance that if your relative has rheumatoid arthritis, you perhaps will have a long-drawn-out risk of the sickness.
Smoking- Cigarette smoking forms your threat of making rheumatoid arthritis, above all in case you have an inborn inclination for developing the ailment. Smoking moreover has all the reserves of being connected with more imperative ailment reality.
Environmental Exposure- Rheumatoid arthritis hazard increments with word related openness to material residue. Also, word related residue openness is moreover connected to a danger of creating antibodies to rheumatoid arthritis, accordingly conceivably accelerating movement of the sickness.
Obesity- People — especially women age 55 and more youthful — who are overweight
or hefty have all the earmarks of being at a fairly higher danger of creating rheumatoid arthritis.
Figure1 shows the MRI of a patient with the affliction.
Figure 1 : A. axial plane; B. coronal plane: synovitis, disintegrations and incendiary growths in the radio-carpal, midcarpal, carpo-metacarpal and metacarpophalangeal joints
Various joints are covered with a covering called the synovium, which oils up the joint so it moves much more with no issue. Precisely, the synovium becomes inflamed, thickens, and produces an excess of joint fluid(synovitis) when you have rheumatoid arthritis. Swelling, damages cartilage, and softens the bone within the joint are caused by synovitis and the inflammatory chemicals released by the immune system . According to the American Society for Surgery of the Hand, deformity and instability is caused by the swollen tissue which stretch the surrounding ligaments(connective tissues that join two bones). The inflammation may also weaken and damage tendons(connective issues that join muscle to bone). The aggravation may moreover disable and harm ligaments. Tendons are connective tissues that join two bones; ligaments are connective issues that join muscle to bone.
Right when RA strikes the hand, it is by and large norm in the wrist and finger knuckles —the MCP (metacarpophalangeal) joint, or the tremendous knuckle where the fingers and thumb meet the hand, and the center knuckle or PIP (proximal interphalangeal) joint considerably more explicitly. Rheumatoid arthritis (RA) impacts joints on the various sides of the body, like two hands, the two wrists, or the two knees. This harmony assists with disengaging it from different sorts of arthritis. The confirmation of rheumatoid arthritis depends from a general point of view on the 1987 adjusted models of the American College of Rheumatology (earlier, the American Rheumatism Association) [72], including clinical, biologic, and radiologic divulgences. The joints of the hands are among the first to be influenced in rheumatoid arthritis, and they are astoundingly persuading in the appraisal of patients with suspected early rheumatoid arthritis [73]. X-shaft has been demonstrated to be more delicate than radiography at seeing bone disintegrations in the hands and wrists of patients with rheumatoid arthritis [74, 75]. In like manner, MRI can give depiction of edema, hyperemia, and joint radiation, identically as perspective on synovial pannus with the usage of IV gadolinium implantation [74–78].Advances in AI, especially CNN and Reinforcement Learning have opened extra open entrances in the field of drug, making structures for seeing dermatological debasements [59], lung pathologies [60], breaks and bone damage [61], among others, to tie the space for give and take in clinical finding and attracting an early revelation of illnesses.
Associated Works
The finish of Periarticular Osteoporosis (one of the consequences of RA) was tried in the evaluation coordinated by Murakami et al. [63] .Thickness characteristics of hand X-transmits utilizing histogram assessment, co-occasion frameworks, fourier changes and extraction of line parts were chosen by the structure .
A changed assurance plan of RA from hand radiographs, utilizing a couple of electronic picture taking care of figuring for feature extraction and a neural association for its strategy was proposed by Chokkalingam and Komathy [64] . Twenty three pictures were used for setting up the model in any case no underwriting tests were performed, along these lines its precision is dull.
Considering the area of bone deterioration, Murakami et al. [65] executed a system for diagnosing RA by using 129 radiograph pictures. In this evaluation, a division assessment and a CNN were used to remove the region of the phalanges and for perceiving the presence of the pathology . 30 cases with RA was taken for the work.The rates of true positive and false positives obtained were 80.5 % and 0.84 % respectively, the fake positive number of the division calculation was 3.3 per case.
The existing systems present restricted results, with high slip up rates and low hypothesis to detect RA.
Made on Convolutional Neural Network, works on retrained model with less number of images, detects simply finger joint deteriorations and it fail to recognize intercarpaljoints. Sensitivity for deteriorations was unmistakably low. Their unflinching quality should improve to be used as an end mechanical gathering in the clinical environment, which has no protection from botches in light of the regular meaning of this assignment.
Methodology
For the improvement of this work ,we used Convolutional Reinforcement Learing Technique to perceive Rheumatoid Arthritis from hand and wrist MRI, considering Feature Extraction, Deep learning and Reinforcement learning methodology to distinguish RA(Rheumatoid Arthritis) to diminish human dependence and takes most choices naturally.
Imagedataset
The pictures utilized in this examination came from Dr Kirubanadan, Radiologist,Noble scancentre,Vaniyambadi,Tamil Nadu ,associated with Apollo Hospitals and Central labs ,Thrissur Kerala with the assistance of Dr. Hari Krishnan G ,Rheumatologist,Kottayam Medical College,Kerala.They comprise of 564 gray scale MRI of both hands and wrist of the patients between the age 30 to 60 , utilized by clinical expert in their analysis of RA. The properties of dataset is displayed in Table 1.
Table 1.Image database characteristics.
Total cases 564
RA-affected 282
RA-not affected 282
Resolution of Image 1378 x 654 pixels
Format- Image DICOM
Bit depth 32 bits
Ethics
It is not possible to expect to relate the patient's name to the MRI.They are anonymous .Keeping all rules of confidentiality ,they were just utilized to build the model .
Software Tools
For the ease, speed and control during the arrangement of complex networks, we used Python, Tensorflow and Keras. OpenCV ,sklearn, imutils,matplotlib,and Numpy modules were used to manage and quantify the photos.
Algorithm
KCP Algorithm
Required import statements for packages: This characterizes all the significant packages for picture pre handling, preparing system, data wrangling and directory storage.
Dataset of 282 Images of RA influenced and another 282 pictures of Normal pictures are stacked in the 'yes' and 'no' indexes separately.
Using the ImageDataGenerator capacity of cv2 (opencv) for making more pictures utilizing picture augmentation, the outcome is increased in addition to the ordinary images.(6200 pictures after augmentation) .These augmented images are utilized to identify the contours of the images for cropping of the image, aka edges of the image. This uses erode and dilate. When the shapes are done, the example picture and the cropped picture is shown.
These pictures are given as 3 dimensional , and the influenced and ordinary pictures are plotted for review of the augmented pictures.
Data is part utilizing train test split into 3 phases – train, approve and test. 70% for preparing, 15
% for validation and another 15% for testing.The state of the pictures, for all x_train, x_val and x_test seen and showed.
The model is planned with (input layer + zeropadding layer + convolutional layer+ maxpooling layers + softmax layer + straighten layer). This is a binary classifier with 1 as Affected and 0 as Normal.
The synopsis of the model, is brought with the state of the dev network made. The tensorboard is instated.
The models is prepared for 30 epochs, the callbacks are utilized for each age which stores the validation accuracy and the age for each model made.
The trained network details are utilized with history which stores all the loss, accuracy and f1 score subtleties of all train, validation and test clumps. The loss and accuracy plots are plotted
The best model is picked from the maximum validation accuracy from the ages , and the measurements are assessed with the hyper parameters.
Finally the after effects of the accuracy, exactness and f1 score are determined for test and validation datasets in the confusion matrix.
Use Deep Reinforcement learning to anticipate and just as train the framework.
Figure 2: Deep Reinforcement learning architecture[79]
Data Augmentation For Rheumatoid Arthritis Detection
The dataset contains 2 organizers: yes and no which contains 564 Hand MRI Images. The folder ‗yes‘
contains 282 Hand MRI Images that are RA influenced and the organizer ‗no‘ contains 282 Hand MRI Images that are Normal.As this is a little dataset, We have utilized information augmentation utilizing ImageDataGenerator to make more pictures on the fly.It is very difficult to train a model with limited data. To overcome this ,we did augmentation of the images.by ransforming the MRI with a random combination of rotation, width shift ,height shift, shear, brightness ,horizontal flip,vertical flip and fill mode .Since the training examples got incremented ,it helped us to improve the classification results of the arcghitecture.The progressions applied, granted the networks to become familiar with these changes in various points and sizes without having influence on the visual highlights in little locales of the primary pictures. After augmentation, we have 6200 images. Each images has a shape of (240, 240, 3)=(image_width, image_height, number_of_channels)
Preprocessing & Segmentation
The dataset contains 2 organizers: yes and no which contains 564 Hand MRI Images. The folder 'yes' contains 282 Hand MRI Images that are RA influenced and the organizer no contains 282 Hand MRI Images that are Normal. After augmentation we have a sum of 6200 Images for the identification.
Figure 3: Original Image and Cropped Image
Load Data
The following function takes two arguments, the first is a rundown of directory paths for the folders 'yes' and 'no' that contain the picture information and the subsequent argument is the picture size, and for each picture in the two directories and does the accompanying:
Read the picture.
Crop the piece of the picture addressing just the hand and wrist.Resize the picture (in light of the fact that the pictures in the dataset come in various sizes (which means width, height and # of channels). Along these lines, we need the entirety of our pictures to be (240, 240, 3) to take care of it as a contribution to the neural organization.
Apply standardization since we need pixel esteems to be scaled to the reach 0-1.
Append the picture to X and its mark to y.
After that, Shuffle X and y, on the grounds that the information is requested (which means the exhibits contains the initial segment having a place with one class and the subsequent part having a place with the other class, and we don't need that).
At long last, Return X and y.
Figure 4 :Plot Sample Images For RA Data Set Division
We used the following way to split:
• 70% of the data for training.
• 15% of the data for validation.
• 15% of the data for testing.
Results and Discussions
Build The Model
• There are 9 layers in the model
Table 2 : Number of parameters
Figure 5 :Number of layers in the Serialization Model Total Parameters 11137
Trainable parameters 11073 Non trainable parameteres 64
Figure 6 :Convolutional Reinforcement Learning Network Architecture for RA Detection Train The Model
For the determination of the optimization algorithm Adam was thought of, contrasting their outcomes after 30 epochs of preparing. Adam was picked for having the quickest combination result.We used Serializing Model which is the best model for industry organization as it decrease measure and improve performance.Figure 6 shows the plot of loss and accuracy.
Figure 6 :Plot Loss & Accuracy
Load The Best Model
After training we evaluated the best model on the testing data by using pretrained model.
Table 3 : Accuracy of the best model on testing data
Result Table 4 displays best validation results and test accuracy.
Table 4 : F1 Score For The Best Model On The Testing Data & Validation Set
Training Data
Validatio n Data
Testing Data
No : of Examples 4340 930 930
No : of positive examples 2184 441 474
Percentage of positive examples 50.32% 47.42% 50.97%
No: of negative examples 2156 489 456
Percentage of negative examples 49.68% 52.58% 49.03%
Table 5 : Result Interpretation
Test Loss 0.5157854557037354 Test Accuracy 0.7935484051704407
Conclusion
In this examination we arranged and evaluated the certified enlightening assortment with Convolutional Reinforcement Learning Techniques. The RA detection is successful with a validation accuracy of 80% and a training accuracy of 99-100% . The test sets have been created and found that the model identifies the images with the expected accuracy and serves the purpose.We propose this system that recognizes RA from hand and wrist MRI without broad preprocessing or excellent features, simply using rough pixel regards and achieving favored exactnesses over near models of the front line. The major results show the ability of this structure.
Acknowledgements
The authors might want to offer their thanks to Dr. Hari Krishnan G ,Rheumatologist,Kottayam Medical College,Kerala and Dr Kirubanadan, Radiologist,Noble scan centre ,Vaniyambadi,Tamil Nadu for supporting this examination .
References
[1] D. S. C. Mathers and B. Pfleger, Criterion 1 (2006). [2]D. Aletaha, T. Neogi, A. J. Silman, J.
Funovits, D. T.Felson, C. O. Bingham, N. S. Birnbaum, G. R. Burmester, [2] V. P. Bykerk, M. D. Cohen et al., Arthritis Rheum. 62, 2569 (2010).
[3] A. Larsen, K. Dale and M. Eek, Acta Radiologica.Diagnosis 18, 481 (1977).
[4] J. T. Sharp, G. B. Bluhm, A. Brook, A. C. Brower,M. Corbett, J. L. Decker, H. K. Genant, J. P.
Gofton, N. Goodman, A. Larsen et al., Arthritis Rheum. 28, 16 (1985).
[5] A. Esteva, B. Kuprel, R. A. Novoa, J. Ko, S. M. Swetter H. M. Blau and S. Thrun, Nature 542, 115 (2017).
[6] M. Anthimopoulos, S. Christodoulidis, L. Ebner, A. Christe and S. Mougiakakou, IEEE Trans.
Med. Imaging 35, 1207 (2016).
[7] J. Olczak, N. Fahlberg, A. Maki, A. S. Razavian, A. Jilert,A. Stark, O. Sko¨ ldenberg and M.
Gordon, Acta Orthop.88, 581 (2017).
[8] G. Langs, P. Peloschek, H. Bischof and F. Kainberger, IEEE Trans. Med. Imaging 28, 151 (2009).
[9] S. Murakami, H. Kim, J. Tan, S. Ishikawa and T. Aoki, Technological Advancements in Biomedicine for Healthcare Applications, (Medical Information Science Reference, 2013).
[10] S. Chokkalingam and K. Komathy, World Academy of Science, Engineering and Technology International Journal of Computer Information Systems and Control Engineering 8, 834 (2014).
[11] S. Murakami, K. Hatano, J. Tan, H. Kim and T. Aoki, Multimed. Tools Appl. (2017).
[12] C. Szegedy, V. Vanhoucke, S. Io ffe, J. Shlens and Z. Wojna, Proc. IEEE, 2818 (2016).
[13] K. Simonyan and A. Zisserman, arXiv preprint arXiv:1409.1556 (2014).
[14] Y. LeCun, L. Bottou, Y. Bengio and P. Ha ffner, Proc. IEEE 86, 2278 (1998).
[15] M. Lin, Q. Chen and S. Yan, arXiv preprint arXiv:1312.4400 (2013).
[16] F. N. Iandola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally and K. Keutzer, arXiv preprint arXiv:1602.07360 (2016).
[17] M. D. Zeiler, arXiv preprint arXiv:1212.5701 (2012). [18]D. P. Kingma and J. Ba, arXiv preprint
arXiv:1412.6980 (2014
[18] Arnett FC, Edworthy SM, Bloch DA, et al. The American Rheumatism Association 1987 revised criteria for the classification of rheumatoid arthritis. Arthritis Rheum 1988; 31:315 –324 [Crossref] [Medline] [Google Scholar]
[19]. Scott DL, Coulton BL, Popert AJ. Long term progression of joint damage in rheumatoid arthritis.
Ann Rheum Dis 1986; 45:373 –378 [Crossref] [Medline] [Google Scholar]
[20]. Klarlund M, Ostergaard M, Jensen KE, Madsen JL, Skjodt H, Lorenzen I. Magnetic resonance imaging, radiography, and scintigraphy of the finger joints: one year follow up of patients with early arthritis—the TIRA group. Ann Rheum Dis 2000; 59:521 –528 [Crossref] [Medline]
[Google Scholar]
[21]. McQueen FM, Stewart N, Crabbe J, et al. Magnetic resonance imaging of the wrist in early rheumatoid arthritis reveals a high prevalence of erosions at four months after symptom onset.
Ann Rheum Dis 1998; 57:350 –356 [Crossref] [Medline] [Google Scholar]
[22] Ostergaard M, Hansen M, Stoltenberg M, et al. Magnetic resonance imaging-determined synovial membrane volume as a marker of disease activity and a predictor of progressive joint destruction in the wrists of patients with rheumatoid arthritis. Arthritis Rheum 1999; 42:918 –929 [Crossref]
[Medline] [Google Scholar]
[23] McGonagle D, Conaghan PG, O'Connor P, et al. The relationship between synovitis and bone changes in early untreated rheumatoid arthritis: a controlled magnetic resonance imaging study.
Arthritis Rheum 1999; 42:1706 –1711 [Crossref] [Medline] [Google Scholar]
[24] Sugimoto H, Takeda A, Hyodoh K. Early-stage rheumatoid arthritis: prospective study of the effectiveness of MR imaging for diagnosis. Radiology 2000; 216:569 –575 [Crossref] [Medline]
[Google Scholar]
[25] https://www.pinterest.com/pin/495747871479893348
[26] M. BETANCOURt-HERNa´NDEZa†, G. VIERA-LO´PEZ b, A. SERRANO-MUN˜OZb,,‖ AUTOMATIC DIAGNOSIS OF RHEUMATOID ARTHRITIS FROM HAND RADIOGRAPHS USING CONVOLUTIONAL NEURAL NETWORKS‖, Article in Revista Cubana de Fisica · July 2018.
[27] Manikandan, R and Dr.R.Latha (2017). ―A literature survey of existing map matching algorithm for navigation technology. International journal of engineering sciences & research technology‖, 6(9), 326-331.Retrieved September 15, 2017.
[28] A.M. Barani, R.Latha, R.Manikandan, "Implementation of Artificial Fish Swarm Optimization for Cardiovascular Heart Disease" International Journal of Recent Technology and Engineering (IJRTE), Vol. 08, No. 4S5, 134-136, 2019.
[29] Manikandan, R., Latha, R., & Ambethraj, C. (1). An Analysis of Map Matching Algorithm for Recent Intelligent Transport System. Asian Journal of Applied Sciences, 5(1). Retrieved from https://www.ajouronline.com/index.php/AJAS/article/view/4642.
[30] R. Sathish, R. Manikandan, S. Silvia Priscila, B. V. Sara and R. Mahaveerakannan, "A Report on the Impact of Information Technology and Social Media on Covid–19," 2020 3rd International Conference on Intelligent Sustainable Systems (ICISS), Thoothukudi, India, 2020, pp. 224-230, doi: 10.1109/ICISS49785.2020.9316046.
