Brain Tumor detection based on MRI Image Segmentation Using U-Net
S. Raghu1 and Prof. T. Adi Lakshmi2
1Research Scholar, Dept. of CSE, Osmania University, Hyderabad
2Professor and Head, Dept. of CSE, Vasavi College of Engineering Hyderabad
1[email protected], 2[email protected]
Abstract: In the quantitative assessment of brain tumors, tumor identification is a serious challenge. Magnetic Resonance Imaging (MRI) has grown in popularity in recent years as a result of its non-invasive and powerful soft tissue comparison capabilities. The practice of using magnetic resonance imaging (MRI) to find brain tumors is quite prevalent. The MRI generates a massive quantity of data. Because of tumor heterogeneity features, manual segmentation takes a long time, limiting the use of accurate quantitative measurements in clinical practice. Manual segmentation is a time-consuming procedure in clinical practice, and its success is greatly dependent on the operator's expertise. Accurate and automated tumor segmentation methods are also required; however, automatic segmentation of brain tumors is challenging due to their significant spatial and structural heterogeneity. This research suggests employing encoder-decoder based convolutional neural networks to completely automate the segmentation of brain tumors.
The paper focuses on UNet which is a semantic segmentation deep neural networks for segmenting tumors from brain MRI images. The networks are trained and evaluated on a publicly available standard dataset, with the Dice Similarity Coefficient (DSC) serving as the measure for the whole projected picture, including tumor and backdrop.
1. INTRODUCTION
Image processing, as the name implies, is a procedure that involves first fragmenting an image into sections with consistent characteristics and then distributing them to a number of classes based on their different attributes. Because medical image processing is an interdisciplinary area, it has attracted many professionals and academics from diverse fields such as applied mathematics, computer science and engineering, biology, medicine, and the like. The more the advancement of technology, the greater the obstacles that emerge in exact proportion. One of the major issues is figuring out how to process and evaluate a large number of pictures for the goal of diagnosing different illnesses and determining the appropriate treatment approach.
The most effective solutions are as effectively implemented as a previous class-based, or rather, class-dependent knowledge of the parameterized models intended for the needed information is deemed necessary. The process advances from the brain tumor as a hard mass initiated by unconstrained cell divisions in the human brain, with irregular cell development (Subhranil Koley et al. 2016) as a consequence. There are two types of tumors: benignor non-cancerous tumors and malignant or cancerous tumors. Although a potentially damaging force is exerted in the benign tumor, its spread into adjacent or adjoining brain tissue is improbable since it is classified as a sluggishly climbing tumor. The latter, a malignant tumor, which is characterized as a fast growing
tumor, on the other hand, is capable of expanding into nearby brain regions. Apart from the two primary forms mentioned above, there is a third type known as a pre-malignant tumor, which is a pre-cancerous stage that is considered an illness, and failure to get correct treatment in this situation would eventually lead to cancer. Tumors harm normal brain cells, creating inflammation that puts pressure on various areas of the brain as well as raising pressure within the skull.
Image segmentation is a technique for detecting objects and borders (lines, curves, and so on) in pictures. To be more specific, image segmentation is the process of assigning a label to all pixels in a picture such that pixels with similar labels have common characteristics. The approaches of segmentation may be divided into two categories: region-based segmentation and border segmentation. One of the most basic and often used methods for area- centered segmentation is region expanding.
Medical Image Processing
Image processing is the process of dispensing images with mathematical operations using any form of image processing in which the input is an image, a series of images, or a video, such as a photograph or video frame, and the output is either an image or a set of features or parameters connected to the image (Kunlei Zhang et al. 2013). In patients with head traumas, CT scanning of the head is often used to identify hemorrhage, brain damage, and skull fractures. A burst or leaking aneurysm in a patient with a sudden strong headache causes bleeding.
The approach for producing computer-assisted diagnostics for brain tumor gradinghas just been accessible in the image processing field (Kimmi Verma et al. 2013). Atlas-based segmentation, probabilistic-information- based segmentation, and machine-learning-based segmentation are just a few of the automated segmentation systems that have been developed. An image is non-linearly recorded onto a manually specified atlas image using the atlas based segmentation technique (Min Hyeok Bae et al. 2010). Picture enhancement is the process of improving the quality of a digital image without having enough information about the deterioration of the original source image.
The technique of segmenting a digital picture into many pieces is known as image segmentation.
The main purpose of segmentation is to make a picture more understandable and simpler to examine by simplifying or changing its representation (Rohini Paul Joseph et al. 2014). Objects and boundaries (lines, curves, etc.) in pictures are often located via image segmentation.
Image segmentation, to be more specific, is the process of giving a label to each pixel in an image in such a manner that pixels with the same label have certain common properties. Image segmentation denotes the image's homogenous parts. Image segmentation methods may be divided into numerous areas, including threshold-based segmentation, edge-based segmentation, region-based segmentation, clustering approaches, matching, and so on. For gray scale photos, several algorithms have been developed. The technique of segmentation for color photos, like grey scale image processing, delivers clear-cut information about the objects in the images. The scientific community, on the other hand, has paid less attention to image processing of colored structure images.
Segmentation Process
The main goal of segmentation is to divide a given picture into several sections of interest. Some segmentation algorithms, like as threshold-based segmentation, accomplish this purpose by searching for borders between sections based on gray scale or color characteristics discontinuities.
The assumption behind region-growing algorithms isthat nearby pixels within a region have comparable values. The most popular method is to compare one pixel to its neighbors. The Statistical Region Merging (SRM) approach begins by creating a graph of pixels using four connections and weighting the edges by the absolute magnitude of the intensity difference. Each pixel makes a single pixel area at first. SRM then sorts the edges into a priority queue and uses statistical predication to determine whether or not to combine the current areas corresponding to the edge pixels.
In each phase of the algorithm, the region expanding approach always acts on closed areas, avoiding the need for further post-processing to restore the borders of unconnected items. The region membership is determined by a number of factors, including image data and previous knowledge of the structures to be divided.
It is important to establish the localization of the point in the area growth technique in order to estimate the pixel intensity; hence, the idea of the neighborhood must be introduced. The majority of the time, four or eight neighbourhoods is evaluated. The seed point for this approach may be chosen manually or from the prior processing. The seed point is the first place to be subjected to growth. The pixels closest to the seed point are selected in more detail, and the homogeneity criteria are confirmed. The quality of segmentation is affected by a sufficient threshold value for the region membership criteria. However, it's a challenge since the region-growing approach is susceptible to noise, which might lead to over-segmentation.
Magnetic Resonance Imaging
The physician may use MRI and computed tomography of the brain to identify supplementary abnormalities from the norm that are indicative of neurological tissue disease. According to current study, most nervous system specialists only possessed pictures of a few cross segments of a cerebrum on a light board at their disposal, and they relied on their intuition to make a diagnosis or determine the effect of therapy solely based on these images.
Self-loader and programmable devices have appeared to aid diagnosis, as the discipline of clinical image processing has grown in popularity. Cerebrum segmentation, for example, allows researchers to not only see but also measure a volume of functional cortical regions. Physically arranging different types of tumors,hematomas, and brain CT and MRI samples solely based on visual assessment is not always achievable. As a result, clinical image segmentation and classification are critical for clinical exploration, analysis, and applications, necessitating the use of solid, dependable, and adaptable division algorithms. The problem is illustrated together with the goals in this section, which looks at numerous types of brain infections that are evaluated for decisions.
An MRI examination allows images to be taken from multiple places at almost every edge, while a CT scan only displays cross-sectional images. As a result, we get progressively itemized data. The distinction between normal and diseased tissue is often easier to see on an MRI scan than it is on a CT scan. CT images are widely used in the studyof ischemic stroke, drain, and hematoma because they are more accessible in a clinical environment, more economical to work with, earlier in the examining time, and more solid, and they are faster to get and comparable to most life-sustaining equipment.
Fig1.MRI sample
Fig 2: Three typical brain tumors: (a) meningioma; (b) glioma; and (c) pituitary tumor.
Because of their claustrophobia, a few people are unable to have an MRI. If a patient has a metallic embed, such as metallic heart valves, aneurysm cuts, or other ferromagnetic material that will be impacted by a solid attractive field, X-ray is also not possible. Patients with severe draining are also unsuitable for MRI examination because blood coagulation forms a tissue that is difficult to distinguish from normal tissue. CT, on the other hand, provides increasingly more geometrical precision in the image. In the emergency department, CT is still the best way to diagnose unaware patients with suspected intracranial drain and hematoma.
2. LITERATURE SURVEY
The tumor boundaries and interest structures are manually sketched or painted in anatomical structures with different labels. Human specialists apply extra knowledge, such as brain anatomy, in manual segmentation, in addition to the information presented in the image. Because of the structural and geographical variability of the tumor part, manual segmentation is prone to errors.
Different neural network-based automated techniques for identifying and segmenting brain cancers using MR images have been suggested in recent publications. The following are the many totally automated brain tumor segmentation approaches that have been discussed.
Image segmentation has been used to identify encephalon tumors, which is a very important step for getting the proper treatment at the right time. Different classifiers, including as FCM, SVM, ANN, and the expectation- maximization algorithm, have been suggested for relegation to extract the most important information from medical imaging treatments. A summary and conclusions from a few studies are offered.
Sergio Pereira et al., (2016) proposed a new neural network for segmentation. Their hypothesis is that by allocating fewer weights in the network, tiny size kernels enable the deep architecture to prevent over fitting. In BRATS 2013 MRI images, they also demonstrated that a preprocessing step of pixel intensity normalization combined with data augmentation is particularly successful for brain tumor segmentation. The author also competed in the BRATS 2015 Challenge and came in second place.
Using a UNET-based deep neural network, Dong et al., (2017) developed a completely automated brain tumor identification and segmentation approach. They demonstrate that their technique can efficiently and reliably segment MRI brain tumor images from the BRATS 2015 benchmark dataset.
Mohammad Havaei et al., (2017) used CNN to segment brain MRI images from the BRATS 2013 datasets using several segmentation topologies such as Two-pathway architecture and cascaded architectures. Their suggested approach takes use of both local and global contextual factors at the same time. The last layer is a convolutional implementation of a fully linked layer that speeds up the process by 40 times. The results of their network's performance on the 2013 BRATS test dataset revealed that it outperformed the published state-of-the-art networks atthe time.
Saddam Hussain et al. (2017) suggested CNN architecture for brain tumor segmentation that combines global and local feature maps. By minimizing features in a totally linked layer and limiting the number of parameters and the likelihood of over-fitting, the employment of max- pooling and drop out layers stabilizes the testing process, boosting training and test speed. Pixel normalization is also employed as a preprocessing step in this case. Minor falsepositives are taken out utilizing morphological techniques during the post processing stage. The investigation is based on the BRATS 2013 brain MRI dataset.
The use of a fully convolutional neural network for automated medical image segmentation did not exhibit sufficiently accurate and consistent results for clinical application, according to Wang et al., (2017), as did the absence of picture-specific adaption and generalization to previously unknown item classes. They developed a novel dynamic segmentation architecture for deep learning that includes CNNs in the bounding box and scripting pipeline to address these difficulties. They also recommended picture contextual fine tuning to adapt the CNN model to a new image that was either unattended or didn't need any further user interactions (supervised).
They conducted their research using the BRATS 2015 data collection.
On tough data such as BRATS 2015 and ISLES 2015, Konstantinos et al. (2017) constructed a dual route, Deep Madic, 3D CNN architecture for automated lesion segmentation that outperformed state-of-the-art. They also demonstrated the advantages of utilizing tiny convolutional kernels in 3D CNNs, decreasing trainable parameters while lowering computing costs, allowing for the development of a more discriminative network. They also recommended using parallel convolutional pathways for multi-scale processing to provide an efficient approach for analyzing big picture contexts.
Dense-Res-Inception Net is a novel deep neural network proposed by Liang Chen et al., 2017.
(DRINet). When numerous parameters such as strength, location, shape, and tumor size fluctuate, the features collected by typical convolution layers are not perfect. DRINet provides a solution to this problem. It comprises of three blocks: a dense link convolution block, a residual inception module deconvolution block, and an unpooling block. In three demanding applications, the suggested design outperforms the U-Net: multi-class cerebrospinal fluid (CSF) segmentation on brain CT images, multi-organ segmentation on abdomen CT images, and multi-class braintumor segmentation on MR images.
Dmitry Lachinov et al., 2018, introduced an autonomous brain tumor segmentation system that addresses the following two issues that emerge while using multimodal MRI scans: complex input and classifier overfitting. The author solves the issue of heterogeneous input by employing several encoders UNET so that each unique input modality creates matching feature maps individually, and then the author describes how to integrate these feature maps. They also demonstrated how to effectively combine many models running at various resolutions to construct a cascade of classifiers. Each subsequent classifier improves the segmentation for its unique size by using the predictions of the prior one. This allows for iterative result refining with fewer parameters than is possible with deep networks. The author utilized BRATS 2018 brain MRI pictures and calculated the average. The validation dataset yielded Dice scores of 0.908, 0.784, and 0.844 for the Whole tumor, Enhancing tumor, and Tumor core, respectively.
Cui et al., (2018) created a unique, completely automated segmentation technique based on in vivo brain glioma MRI data. Their approach can not only pinpoint the tumor's location, but also segment the intratumor structure. There are two sub networks in the proposed deep cascaded CNN:
(1) a tumor localization network (TLN) and (2) an intra tumor classification network (ITCN). The first sub network's purpose was to use the MRI to pinpoint the tumor's location. After that, the ITCN was utilized to divide the tumor area into numerous sub- regions. The suggested technique was verified for multimodal brain tumor segmentation (BRATS 2015) data sets, with favorable segmentation results and quicker segmentation speeds.
U-NET-based deep neural networks were used by Hai Thanh Le et al., 2020, to classify tumorous tissues into four categories: necrosis, edema, non-enhancing, and enhancing tumor. The Brain Tumor Segmentation Challenge 2013 (BRATS 2013) dataset was utilized in their experiments.
With the metric values Dice and sensitivity of 0.83and 0.85, they calculated the best performance
score.
Mostefa Ben Naceur et al., 2020 created a three-stage pathway to improve the prediction of tumor areas in Glioblastomas (GBM). In the first stage, they developed deeper Convolutional Neural Networks (CNNs), then extracted multi-dimensional features from higher-resolution representations of CNNs in the second stage, and fed the extracted characteristics of CNNs into various traditional machine learning algorithms such as Random Forest (RF), Logistic Regression (LR), and principal component analysis (PCA-SVM). They worked with the BRATS- 2019 dataset. For the total tumor, tumor core, and enhancing tumor, the average Dice score of their pipeline was 0.85, 0.76, and 0.74, respectively. Intensity-predicted categorization of MR images intra-scan in homogeneities approach with application of the EM algorithm to a completely automated procedure for tissue segmentation from MRI data was proposed by W. M. Wells et al.
(1996).
Yannis A. Tolias et al. (1998) proposed a flexible fuzzy clustering method for image segmentation with spatially variable sample vectors for various window dimensions, based on amultire solution representation.
Furthermore, Yannis A. Tolias et al. (1998) suggest a novel method for improving fuzzy clustering results by imposing spatial restrictions for addressing picture segmentation issues. The author has developed a Sugeno-type rule-based scheme with three inputs and 11 rules that works in conjunction with the clustering results generated by the popular FCM and/or PCM algorithms to provide excellent picture partitions in terms of region softness and noise reduction. By using stochastic field modeling, the outcomes of the proposed rule-based neighbourhood development (RB-NE) system are compared to well-known segmentation techniques.
Dzung L. Pham et al. (1999) introduced fuzzy image segmentations that were constructed by modifying the impartial function in the FCM algorithm, allowing each class to vary throughout the picture via the centroids.
A.W.C.Liew et al.(2000) used a 3 × 3 window to use spatial background information. The technique is consistentwith the image in terms of logic, since nearby pixels are hidden in non- homogeneous portions of the picture. A cluster merging solution is proposed that combines two clusters based on their proximity and overlap level. The suggested method lowers noise and is more competent at identifying relegation ambiguity and duplication with varied cluster assortment than the prognosis able FCM Algorithm, according to research findings using both synthetic and real pictures.
Mohamed N. Ahmed et al. (2002) introduced a method that recompenses by the label in its immediate neighbourhood by altering the goal functions of the usual FCM algorithm. This functions as a salt and pepper noise regularizer.
Dzung L. Pham et al. (2002) proposed a novel way to fuzzy clustering that uses the consequence term to produce a constant algorithm. A criteria based on cross-validation is used to determine the strength of the consequence term, which decreases picture noise.
Yanling Li et al. (2010) proposed a fast FCM clustering method with spatial restrictions (FFCM-S)
to overcome the problem that the FCM-S technique was too slow. The FFCM-S method was used since it has a higher convergence rate.
Damodharan et al. (2016) suggested a neural network-based encephalon tumor perception and relegation approach. Using NN predicated analysis, the quality rate is created separately for sectionalization of grey matter (GM), white matter (WM), and cerebrospinal fluid (CSF), as well as tumor area, and claims an accuracy of 83 percent.
To increase the precision of the classifier, Salem et al. (2016) used SVM, fast Fourier transform (FFT), and the Minimal Redundancy-Maximal-Relevance (MRMR) technique. This approach had a 98.9% accuracy rate. For feature removal from MR images, Chaddad et al. (2015) used the Gaussian mixture model (GMM). Using PCA and wavelet-based characters, the performance of the GMM feature extraction is increased, and "accuracy" of
97.05 percent for "T1-weighted" with "T2-weighted" and 94.11 percent for "FLAIR weighted MR"
images is obtained. To assess segmented grey matter, Zanaty et al.(2012) [44] suggested a hybrid FCM and Jaccard similarity coefficient approach. Kumar & Vijaya kumar et al(2015) [45] use PCA and SVM to get a similarity index of 96.20 percent. Cui et al. (2013) proposed a localised fuzzy clustering approach that uses the Jaccard similarity index as a measure of segmentation correctness and claims an accuracy range of 83 to 95 percent. Wang et al. (2014) developed a segmentation approach based on the intensity in homogeneities found in image partition. Sachdeva et al. (2013) used a collection of 428 MR images to perform multiclass brain tumor relegation, segmentation, and feature extraction using a dataset of 428 MR images. The use of ANN and PCA- ANN in this system resulted in an increase in relegation accuracy of 77 percent to 91 percent.
Nilesh Bhaskar rao Bahadureet et al. (2017) acquired 96.51 percent accuracy, 94.2 percent meticulosity, and
97.72 percent sensitivity using dice similarity catalogue, which is one of the major elements in determining the accuracy segmentation and support vector machine for classification.
Azian Azamimi Abdullah et al. (2012) used a cellular neural network (CNN) method to diagnose a brain tumor in MRI images using a Graphical User Interface (GUI) in MATLAB. The CNN simulator was able to successfully locate the location of a brain tumor in a shorter amount of time and provided a 100 percent confirmation of the tumor's presence.
Ramzi A.Haraty et al. (2015) introduced an upgrade to k-means clustering that can cluster large amounts of data and uses a distributed variation of the G-means technique to boost processing capacity.
DinhThuan Nguyen et al. (2013) proposed an improved technique for learning k means clustering in order to increase performance based on speed.
Christos Boutsidis et al. (2015) provided a research on dimensionality reduction for k-means clustering, which results in a minimal number of features and constant-factor estimate for k-means objective assessment. For k- means clustering, Mingjun Song et al. (2015) presented three stable
approximation techniques. They proposed three O(1)- approximation algorithms whose run time is self-determining for the k-median difficulty in this research.
EffatNaaz et al. (2016) used the K-means algorithm to get alternative perspectives of medical notes, increasing their visibility. This finding emphasises the accuracy of drugs used to treat symptoms. A modified K-Meanss algorithm was provided by Leonard WafulaWakoli et al. (2014) to warn out suspicious assertions.
By training a Deep CNN model, Kai Zhang et al. (2017) shown excellent usefulness in image denoising duties. Regarding multi parametric MR data retrieved from follow-up GBM victims, Adrian Ion-Margineanu et al.
[58] discussed on establishing a classifier that divides between tumor succession and regression.
It also calculates BER for each time point and wBER for each time point to determine performance.
William Thomas H.M (2015) offers an MRI picture for tumor identification utilising morphological filtering, which is eliminated from the input image and displayed using morphological operations and segmentation algorithms. The experiment was carried out to identify tumors utilising morphological image processing and subsequently segmentation in a specific watershed for a canned MRI picture of the human brain for several samples. Sayali, V Roeli, et al.
(2015) demonstrated how to extract information and numerous attributes from segmented pictures for brain tumor identification. Garima Singh, et al. (2016) offered a well-organized Nave BayesClassifier and Support Vector Machine (SVM) for reliable prediction and classification, as well as a novel approach for recognising brain cancer utilising normalised histogram and segmentation using K-means clustering technique. To remove unnecessary noise and recover the tumor from the MR image, Nabizadeh et al. (2015) developed aunique approach that included Normalization of Histogram combined with K-means Segmentation. Krupali D.Mistry et al. described a technique for identifying Enchondroma bone cancer from MRI images using imageprocessing, segmentation clustering approaches, and FCM clustering.
Two strategies for planning the picture in MRI were described by Priya Patil et al. (2012). This design aid shows how to discern the borders of brain cancer and determine the true tumor size, as well as how to use the f- transform to provide precise information such as reassembling lost edges and removing silent edges. In an MRI image, accuracy and simplicity are interdependent.
KailashD.Kharat, et al. (2012) suggested a method for segmentation, quality vector extraction, and model learning that combines Multi-Layer Perceptron (MLP) and Radial Basis Function (RBF) methods. These two approaches identify suspicious areas that have similar characteristics to tumor areas.
In the areas of large-scale retrieval, Zhongyu Li et al. (2017) provided the assessment set of rules with a range of investigative and diagnostic situations, which may enhance the enactment of medical picture investigation. T. Sathies Kumar et al. (2017) presented encephalon tumor detection, which allows confining a mass of abnormal cells in a slice of MR using "SVM Classifier" and segmentation to know about the dimension of the tumor in that segmented area, as
well as training the extracted features of the segmented portion using ANN to display the tumor category.
Emre Dandi et al. (2017) created software with a user interface specifically for this purpose. As a result, the case of doctors being guilty will be presented. As a result, the application software will reduce mistakes and might be usedas a supplementary method of brain tumor segmentation. It has been shown in comprehensive test studies using picture datasets that the suggested application can effectively identify brain cancers.
Aqhsa Q, et al. (2014) provided classification for MLP utilizing energy, entropy, contrast, and a few additional statistic characteristics such as mean, median, variance, and association, as well as a feature selection approach to reduce the feature space by employing neural networks.
M.Jasmine, et al.(2012) reported the examination of tumor segmentation in every year that is established for brain cancers that may be readily recognised by MR images and demonstrated that the degree of segmentation accuracy would enable diagnosis and analysis. They also looked at MR pictures of encephalon tumor segmentation methodologies and imaging modalities, as well as how the procedures go before segmentation is assessed by comparing different brain images and taking the advantages of various ways into consideration.
Nicolas Sauwen et al. (2015) described a method for detecting tumor substructures (i.e. potential tumor, necrosis, and edemas) in twenty-four glioma patients. The presence of distinct tissue types makes it difficult to identify tissue- specific trends among patients, especially when mixed "tissue"
classes are evaluated.
Neha Rani, et al. (2016) used a neural network with back propagation to classify the performance of tumor components, demonstrating that this method produces excellent accuracy and apperception while also reducingtime spent.
Sanghamitra T. Kamble, et al. (2015) showed several forms of cancers and used the K-means method to extract information from the encephalon cell. For accurate brain tumor partitioning, the noise-free picture is supplied as an input to the "k-means" and tumor.
WeidongJi, et al. (2016) presented the fundamental principles of RBF, then comprehensively accounts the RBF domestic and overseas study status, discusses the superiority and disadvantages of various methods, and compiles the explanation, as well as analyses current problems and future development trends. Anisotropic dissemination model for image enhancement and K-means clustering approach for categorising tissues were presented by M. Masroor Ahmed et al. (2015).
Appreciative findings were obtained using the suggested approach, which took into account weighted grey level intensity photographs.
Piyush M. Patel et al. (2013) introduced a cluster research approach that uses the K-means algorithm, notably forlarge volume datasets in partitioning to multi-sampling adoption.
3. PROPOSED METHODOLOGY
CNN-based classifier networks, such as AlexNet, VGGNet, and GoogLeNet, outperform the ImageNet classification task. To complete the image segmentation process, these networks are now configured by categorizing pixels. Although the aforementioned networks do not perform equally well, the principle of down- sampling up-sampling utilizing pooling and unpooling layers makes the segmentation process relatively easy. The picture is divided on a pixel-label basis in semantic segmentation, which is the process of associating each pixel in an image to a classlabel.
These labels may be anything from a person to a vehicle to a flower to a piece of furniture to a tumor. Autonomous cars, human-computer interface, robotics, and picture editing/creativity tools are just a few of its main uses. On each layer, the deep neural networks are trained from end to end, pixel by pixel.
UNET:
This is very popular semantic segmentation network for biomedical imaging proposed by Olaf Ronneberger in May 2015. It consists of three parts i) The contracting/downsampling path ii) Bottleneck iii) The expanding/upsampling path. This architecture is considered as extension of Fully Convolutional Network in a way that, i) UNET is symmetric, ii) the skip connections between the contracting path and the expanding path apply a concatenation operator instead of a sum. While upsampling, the skip connections are accountable for proving local information to global information. The upsampling approach has a high number of feature mappings because to its symmetric structure, allowing information to be transferred.
The contract/downsampling route is made up of four blocks, each with two 3x3 Convolution Layers + Activation functions and a 2x2 Max Pooling layer. At each pooling layer, the number of feature maps doubles, beginning with 64 for the first block, 128 for the second block, and so on.
The goal of this contracting route is to gather the context of the input picture, which will subsequently be given to the upsampling path through skip connections.
Bottleneck: This segment of the network connects routes that are decreasing and increasing. It is made up of simply two convolution layers (with batch normalization).
The expanding/upsampling route is made up of four blocks, each of which has a deconvolution layer with stride 2, a concatenation with the equivalent cropped feature map from the contracting path, and two 3x3 Convolution layers with activation functions. The goal of this expanding route is to allow for a precise location while keeping the contracting path's qualitative characteristics
Fig 3: The U-architecture net's is seen below. A map of a multi-channel function is represented by the boxes. At the top of the box, the number of channels is indicated. The x-y-size is visible at the box's bottom left border. The white boxes represent feature maps that have been replicated. The arrows represent the different operations.
4. RESULTS AND DISCUSSION Data Set:
To train and evaluate the UNET segmentation network, we utilized a publically available brain tumor dataset of 3064 brain MRIs from 233 individuals are included in the dataset. Meningioma (708), pituitary(930), and glioma are the three forms of brain cancers included (1426).
Preprocessing
To save time and money, the extracted pictures and ground truths are reduced to 256 X 256 pixels.
Colored picture data is taken into account (RGB channels). We separated the dataset's 3064 images into 80 percent training and 20 percent testing data. As a result, there are 2451 images in the training set and 613 images in the test set. Augmentation is used to minimize over fitting and to provide a big training dataset. The addition of information inside the training pictures enriched the training images in this experiment. Data augmentation techniques such as rescale, flip, rotation, zoom, shear, brightness, and others with different parameters are used in this methodology to create a new bigger training set, as illustrated. As a result, the training data set has grown to 10 times its initial size, resulting in a total of 24510 training images.
UNet was implemented using KERAS, which used TensorFlow as the backend in Python. NVidia TESLA V100 graphics card with 16 GB of dedicated memory and 192 GB of DDR4, 2666 MHz and Intel Xeon SKL G-6148 CPU were used to train the networks over a number of epochs. The network's hyper parameters, such as learning rate, batch size, and number of epochs, are fine- tuned by trial and error. Adam [30] was used as the training optimizer, with a starting learning rate of 0.0001. Adam, a variant of the stochastic gradient descent (SGD) method, keeps track of each network weight's learning rate and adjusts it individually as learning progresses. When the
parameter stopped improving, we utilized the Keras callback "ReduceLROnPlateau" to adaptively lower the learning rate. After learning has reached a halt, models benefit from a tenfold drop in learning rate. This callback monitors the quantity, and if the "patience"-number of epochs does not change, the learning rate is slowed. Fig 4(a), (b),(c) show the brain tumor segmentation output by U Net architecture.
Fig 4(a) Brain MRI Image 4(b) Ground truth segmentation 4(c) Predicted Segmentation Evaluation Metrics:
Accuracy: The ratio of correct predictions to the total number of predictions produced by the classifier is known asclassification accuracy. The equation may be used to demonstrate this.
Accuracy =
𝑇𝑃 + 𝑇𝑁
3𝑇𝑃 + 𝑇𝑁 + 𝐹𝑃 + 𝐹𝑁
Where, the number of genuine positive pixels, or pixels that are correctly diagnosed as tumor area, is represented by TP. The number of true negative pixels, i.e. pixels that belong to the backdrop and are correctly designated as background, is represented by TN. The number of false positives, or pixels incorrectly categorized as tumor pixels, is denoted by FP, while the number of false negatives, or pixels corresponding to the tumor area incorrectly classified as background, is denoted by FN.
Precision : Positive incidences as a percentage of total expected positive instances. The model prediction made as positive from the whole dataset is the denominator. Assume you're trying to figure out 'how much the model is correct when it claims it is correct?' It's utilized to locate the tumorous pixels themselves.
Precision = 𝑇𝑃 𝑇𝑃 + 𝐹𝑃
Training performance
The UNET and CNN networks are tested on a 613-image Test dataset. For UNET and CNN, the average DSC of the testing dataset is 0.76 and 0.67, respectively, with an accuracy score of 0.90 for both.
Because it includes the properly categorised background region, the segmented image's accuracy and precision are generally greater than the DSC. Furthermore, it should not sensibly reflect the model's predictive performance since it suffers from the accuracy paradox (which states that a trained model with a given degree of accuracy might have greater predictive power than models with higher accuracy). As a result, in addition to accuracy and precision, we place a greater weight on DSC when studying segmentation networks.
4. CONCLUSION
Brain tumor segmentation is a difficult process that is still carried out by people. Automatic segmentation, on the other hand, is in high demand, as shown by deep learning research. In this research, we explain how to employ semantic segmentation networks, such as UNET, to segment brain tumors automatically. On the figshare brainMRI image dataset, we found that UNET offers superior segmentation results.
By partitioning the whole picture into a number of tiny patches in the training dataset, we hope to increase the performance of both networks even more. To further assess the segmented pictures, we should specify the dice similarity coefficient separately for tumor and background in the future.
It is also possible to investigate the impact of extending encoder decoder blocks in each network on segmentation results. We strive to categorize the kind of tumor in one of three classes after effective segmentation with high DSC: glioma, meningioma, and pituitary tumors.
5. REFERENCES
[1] A. de Brebisson and G. Montana, “Deep neural networks for anatomical brain segmentation,” in CVPR Workshops, 2015, pp. 20–28.
[2] M. Avendi, A. Kheradvar, and H. Jafarkhani, “A combined deep-learning and deformable- model approach to fully automatic segmentation of the left ventricle in cardiac MRI,”
Medical Image Analysis, vol. 30, pp. 108–119, 2016.
[3] H. R. Roth, H. Oda, Y. Hayashi, M. Oda, N. Shimizu, M. Fujiwara,K. Misawa, and K.
Mori, “Hierarchical 3D fully convolutional networks for multi-organ segmentation,” arXiv preprint:1704.06382, 2017.
[4] O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in MICCAI, 2015, pp. 234–241. I. Razzak, M. Imran, G. Xu, Efficient brain tumor segmentation with multi scaleancer statistics two pathway- group conventional neural networks. IEEE J. Biomed. Health Inf. (2018).
https://doi.org/10.1109/JBHI.2018.2874033
[5] N. Abiwinanda, M.Hanif, S.T.Hesaputra, A.Handayani, T.R.Mengko, Brain tumor classification using convolutio nal neural network. In World Congress on Medical Physics and Biomedical Engineering, pp. 183–189. Springer (2019)
[6] T.A. Abir, J.A. Siraji, E. Ahmed, B. Khulna, Analysis of a novel MRI based brain tumor classification using probabilistic neural network (PNN). Int. J. Sci. Res. Sci. Eng. Technol.
4(8), 65–79 (2018)
[7] Arshia Rehman1 · Saeeda Naz1 · Muhammad Imran Razzak2 ·Faiza Akram3 · Muhammad Imran4A Deep Learning-Based Framework for Automatic Brain Tumors Classification Using Transfer Learning, Circuits, Systems, and Signal Processing https://doi.org/10.1007/s00034- 019-01246-3
[8] Yao J. Image processing in tumor imaging. New techniques in oncologic imaging; 2006. p. 79 –102. Sergio Pereira, Adriano Pinto, “Brain Tumor Segmentation Using Convolutional Neural Networks in MRI Images”, IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 35, NO. 5, MAY 2016
[9] Dong H, Yang G, Liu F, Mo Y, Guo Y (2017). Automatic brain tumor detection and segmentation using U-net based fully convolutional networks. MIUA, 3, 1-12.
[10] Mohammad Havaei, (2016) “Brain Tumor Segmentation with Deep Neural Networks”, Preprint submitted to Medical Image Analysis
[11] Saddam Hussain, Syed Muhammad Anwar, “Brain Tumor Segmentation using Cascaded Deep Convolutional Neural Network”, 978 -1- 5090-2809-2/17/$31.00 ©2017 IEEE
[12] Wang G, Li W, Azuluaga M, et al (2018). Interactive medical image segmentation using deep learning with image specific fine tuning.IEEE Trans Med Imaging, 37, 1562- 73 [13] Konstantinos KK, Ledig C, Virginia, FJ, et al (2017). Efficient multi scale 3D CNN with
fully connected CRF for accurate brain lesion segmentation. Med Image Analysis, 36, 61-78 [14] Liang Chen, Paul Bentley, “DRINet for Medical Image Segmentation”, DOI
10.1109/TMI.2018.2835303, IEEE Transactions on Medical Imaging
[15] Dmitry Lachinov, “Glioma Segmentation with Cascaded Unet”, arXiv:1810.04008v1 [cs.CV] 9 Oct 2018 Page 15/17
[16] Cui S, Mao L, Jiang J, Liu C, Xiong S (2018). Automatic semantic segmentation of brain gliomas from MRI images using a deep c ascaded neural network. J Healthcare Engineering, 2018, 1-14.
[17] Le H.T., Pham H.TT. (2020) Brain Tumor Segmentation Using U-Net Based Deep Neural Networks. In: Van Toi V., Le T., Ngo H.,Nguyen TH. (eds) 7th International Conference on the Development of Biomedical Engineering in Vietnam (BME7). BME 2018. IFMBE Proceedings, vol 69. Springer, Singapore. https://doi.org/10.1007/978- 981-13-5859-3_7 [18] Ben Naceur M., Akil M., Saouli R., Kachouri R. (2020) Deep Convolutional Neural Networks
for Brain Tumor Segmentation: Boosting Performance Using Deep Transfer Learning:
Preliminary Results. In:
[19] Crimi A., Bakas S. (eds) Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. BrainLes 2019. Lecture Notes in Computer Science, vol 11993. Springer, Cham.
https://doi.org/10.1007/978-3-030-46643-5_30
[20] Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pp.
1097–1105.
[21] Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large- scale image recognition. In Proceedings of international conference on learning representations. arXiv:1409.1556
[22] Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, (2015). Going deeper with convolutions. In Computer vision and pattern recognition (CVPR). arXiv:1409.4842
[23] Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In International conference on medical image computing and computer-assisted intervention (pp. 234–241). New York: Springer.
[24] Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation (2014), arXiv:1411.4038 [cs.CV]
[25] Badrinarayanan, V., Kendall, A., & Cipolla, R. (2015). Segnet: A deep convolutional encoder-decoderarchitecture for image segmentation. arXiv preprint arXiv:1511.00561 [26] Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale
image recognition. In Proceedings of international conference on learning representations.
arXiv:1409.1556
[27] J. Cheng, Brain tumor dataset. figshare. dataset.
https://doi.org/10.6084/m9.figshare.1512427.v5. Accessed 30 May 2018
[28] Wang, J., & Perez, L. (2017). The effectiveness of data augmentation in image classification using deep learning. Tech. rep., Technical report.
[29] Kingma, D., & Ba, J. (2015). Adam: A method for stochastic optimization. In Proceedings of the 3rd international conference on learning representations (ICLR).
[30] Ajay Mittal, Rahul Hooda, Sanjeev Sofat, “LF-SegNet: A Fully Convolutional Encoder–
Decoder Network for Segmenting Lung Fields from Chest Radiographs”, Wireless Personal Communications,Issue 1/2018.
