• Nu S-Au Găsit Rezultate

View of Cardiac Segmentation from MRI images using Recurrent & Residual Convolutional Neural Network based on SegNet and Level Set methods

N/A
N/A
Protected

Academic year: 2022

Share "View of Cardiac Segmentation from MRI images using Recurrent & Residual Convolutional Neural Network based on SegNet and Level Set methods"

Copied!
10
0
0

Text complet

(1)

Cardiac Segmentation from MRI images using Recurrent & Residual Convolutional Neural Network based on SegNet and Level Set methods

Mikkili Dileep Kumar (Author 1)1*, Dr Kv Ramana (Author 2)2

1Assistant Professor, Information Technology, GEC, Gudlavalleru , India

2Professor, Computer Science and Engineering, JNTUK, Kakinada, India

*MIKKILI DILEEP KUMAR: [email protected]

* Dr. K V RAMANA:[email protected]

Abstract

In recent years, semantic segmentation with Deep Learning (DL) is popularly foundin many real-world applications. It is also specific, mentioned that these techniques are regularly applied to various segmentation and classification techniques in the medical field. The most popular deep learning techniques, especially SegNet and U-Net, are the most used for this type of medical application.Generally, in case of U-Net architecture which can be used with a skip connection and is capable of retrieving fine data during training. However, such a network consumes a lot of computation time compared to other networks. But SegNet is another network used to retrieve the desired information with a computing efficient. Inspired by the work, the skip connection is introduced into SegNet using the residual neural network (ResNet). ResNet consists of a layer and it has taken inputs involving multiple layers of the neural network, giving precise performance. This article offers first a recurrent convolutional neural network (RecNet) based on Seg-Net called R-SegNet and also a recurrent residual convolutional neural network (R2Net) based on SegNet models called R2-SegNet respectively. The strength of SegNet, RecNet and ResNet are used and have produced the architectures to perform the segmentation of cardiac MRI images. There are a number of advantages derived from the proposed architecture. First of all, using a residual unit, proposed architectures were used to carry out training in deep architecture. Second, the use of recurrent residual convolutional layers ensures that the relevant features retrieved to perform the segmentation tasks. Third, the proposed architecture has designed a good SegNet with a limited network parameters and also produce better performance for performing the task of segmentation in cardiac images. In addition, then, applied the level definition method, to extract the contours or surfaces of the cardiac MRI images.The results proves that the hybrid proposed deep learning methods successfully segments the images and also achieves better accuracy compared to standard architectures.

Keywords:

Cardiac segmentation, residual neural networks, recurrent networks and set of levels 1. Introduction

In recent times, effective diagnosis of heart ailments is done through application of cardiac magnetic resonance imaging (MRI) technology in conjunction with standard cardiac assessment protocols[1,2,4,5]. In the cardiac imaging arena, cardiac image segmentation is regarded as the earliest and most reliable procedure. In general, the cardiac image is

(2)

segmented into right and left atrium (RA) (LA) and left and right ventricle (LV). New techniques and algorithms based on the deep neural network (DNN) and the convolutional neural network (CNN) play a significant role in the diagnosis of computer aids in the process of cardiac MRI, in particular in the segmentation of the left ventricle ( LV) and right ventricle (RV) [16] due to the advancements in artificial intelligence and machine learning space. This work mainly aims to introduce proposed hybrid CNN architecture for cardiac segmentation task using standard networks include SegNet, U-Net, RecNet and ResNet .It also analyses the advantages of all previous models, before designing hybrid architectures using standard CNN models so that segmentation of the cardiac MRI images in theleft ventricle (LV) and right ventricle (RV) is done effectively.TheSegNet combines two Convolutional layers having thefilter size 3 *3 with stride and padding .Next, from the every convolutional layer corresponds Batch normalization to perform the normalize channels of the extracted features.

Finally, the rewired linear unit (ReLU) layers helps produce zero inputs out of the negative input without disturbing related dimensions.

Fig1.Overall Architecture of Segnet

The overall architecture of SegNet illustrated in Figure 1. Next, U-Net is another popular deep learning network used for segmenting medical images [16]. The network is worth of retrieving better features from the corresponds encoder to the decoder with the usage of a skip connection . However, it demands for the higher computation power compared to the SegNet. Moreover, all retrieved features are sent across the oversampling convolution blocks in the decoder by using skip connection[18].

This article focuses on hybrid CNN architecture using SegNet, ResNet and Recurrent Neural Network (RecNet) which are respectively called R-SegNet and R2-SegNet. The steps involved in segmenting the heart from MRI images as exhibited in Figure 1.However, in pre- possessing,2D images are extracted from the given 3D MR images. Later send resultant pre- processed images to hybrid CNNmodel for the training and resulted as the segmented image as a post-processing method applied Level Set to extract the regions.

Fig. 2.Functional diagram of the proposed CNN hybrid.

The complete contributions regard this work described as:

(3)

1) Hybrid models called R-SegNet and R2-SegNet are introduced for image segmentation by cardiac MRI.

2) Contours or regions extracted from the results trained using the level set method 3) Experiments are carried out on left and right ventricle cardiac data sets.

4) Comparison with recently proposed advanced methods which show superior performance compared to traditional networks.

The organisational pattern of the present work is given here under: Section 2highlights the relevant previous contributions in this area. The R-SegNet and R2-SegNet models architecture is shown in section 3. The post processing is described in Section 4. Next, in Section 5 explains the data sets, experiment setup as well as the final results. Finally, the conclusion and future directions are mentioned in the final part i.e. Section 6.

2. Related works:

The number of modelshas been proposed in recent years and are already shown that set of deeper networks which are suitable for medical image recognition and segmentation [15].

Moreover, the CNN-based segmentation models using FCN proved better performance for medical image segmentation [10]. The fully automatic heart segmentation methods based on CNN have been proposed [9]. A CNN method combined with a deformable model to perform LV segmentation [17]. The model to manage the segmentation of the left ventricle for MR SAX cardiac images [ 16]. The experimental results showed that CNN-based methods are becoming the new state of the art in the cardiac MRI segmentation [9,10,19]. Later, using recurrent neural networks (RecNet) improved FCN performance especially in large data sets [21,22]. The popular deep learning approach for medical image segmentation is called U-Net [18]. More recently, many researchers have tried to use U-Net [6], which is another symmetrical vgg-type model, to segment LV / right ventricle / myocardium. In addition, to segment cardiac MRI images of the left ventricle proposed a path-based CNN [16]. A method which used to integrate CNN and a recurrent neural network (RecNet) in two phases end- diastolic and end-systolic [23,24]. The method of segmentation using FCN and used for training and verifying on large Cardiac MRI image and the results are promising.

3. R-SegNet and R2-SegNet architectures

The proposed new models called R-SegNet and R2-SegNet are inspired from the number of deep neural network modelResNetRecNet and UNet and used for the segmentation tasks. The strength of the mentioned models are used by proposed learning network techniques. The model which is used RecNet and its variants proven to be best on object recognition.A hybrid model called R-SegNet is made with the combination of ResNet and SegNet. However, a skip connection is introduced in SegNet from the ResNet and such kind of methods are popularly used in the image segmentation. Generally, skip connection from the ResNethelps to retrieve the information on the oversampling path from the information captured in the corresponding encoder element by element addition.

Let's take,Xlbe the input image in the concernlth layer of the R-SegNetunitand corresponding imagepixel at( , )i j , animage sample in kthfeature map in the recurring layer

(4)

(RL). Suppose further the exit from the networkOijkl ( )t is not in time t. The result of the output image to be represented in eq. (1).

( , ) ( , )

( ) ( ) * ( ) ( ) * ( 1)

l k T f i j k T r i j

ijk f l r l k

O tW X tW X t b (1)

From the (1),Xlf i j( , )( )t and Xlr i j( , )(t1)are considered to be the inputs of the traditional convolutional layers atlth recurring layer. Moreover, the Wfk and Wrkare values of weightscorresponds to convolutional layer at kthfeature map, and bias to be represented as

bk.The output of RL from the activation function of ReLU „ f ‟ to be defined as : ( l, l) ( ljk( )) max(0, ljk( ))

F X Wf O tO t (2)

Here, F X W( l, l), denotes thecorresponding outputs of lthlayer related to the R-SegNet unit.

The function F X W( l, l) also used for the both sub-sampling and oversampling layers in the convolutional coding and decoding units of the R-SegNet model. The model of R2U-Net, outputs pass through the residual unit and is presented in Figure3 (d).

Next, the output of the R2-SegNet block is to be represented as xl1and to be evaluated as:

1 ( , )

l l l l

X XF X W (3)

From (3), Xl considered as the R2-SegNet input samples and its output Xl1retrieved from the immediately following sub-sampling or oversampling layers in the convolutional coding and decoding units of R2-SegNet.Moreover, the number of features retrieved from the parameters is same as in the R2-SegNet block shown in Figure 3 (d).

Fig.3.The process model

The basic SegNet model is shown in Figure 3 (a). Second, SegNet with residual connectivity is used, which is often called R-SegNet illustrated in Figure 3 (c). Next,SegNet with recurrent convolutional layers is given in Figure 3 (b). Atlast, the SegNet with recurrent convolutional layers and residual connectivity is illustrated in Figure3 (d),and is named as R2-SegNet.The proposed R-SegNet and R2-SegNet architectures build with equal set of network parameters which are shown with U-Net and ResNet, and also proven from the experimental results both methods produce better results on segmentation tasks. Moreover, it is observed that both Recurring and residual tasks do not raisethe number of network settings, thus significantly affectingtraining as well as test results.

4. Post-processing using the level definition method

The level set method, first applied as a digital technique to track interfaces and shapes [12], have larger applications in image segmentation in recent years. The level set function is

(5)

nothing but the contours or surfaces projected as the zero level set of a higher dimensional function.

Let be image domain, and I: Ra grayscale image. In [12], the imageI is segmented through an outline C that slices image domain in disjoint regions

1, 2,..., N

   , as well as a smooth piece-wise function bringing image closer I , besides being smoothly ensconced in each regioni Through this, the following problem is thus minimized:

2 2

1 2 1 2

( , , ) ( ) ( ( )) ( ) (1 ( ( ))) ( ( ))

FC C I x C Hx dx I x C Hx dx v Hx dx

 

  

(4)

orHis the Heaviside function, and  is a level definition function, whose zero level contour

: ( ) 0

Cxx  partition the image domain in two separate regions 1

x: ( ) x 0

and 2

x: ( ) x 0

. The first two terms of (4) are the data adjustment terms, while the third term, with a weightv> 0, regularizes the zero-level contour. Image segmentation is therefore obtained by finding the level definition function and the constants C1 and C2 which minimize energyF( , C C1, 2).[8]This model is a constant part model (PC), because it assumes that the imageI can be approximated by constants C1 and C2in the regions 1and

2, respectively their membership functions defined byM1( ) H( ) andM2( )  1 H( ) .

“The level set function ,the vector of constants c(C C1, 2,...,CN) and the bias field bare the energy variables , which can therefore be written ( , , )c b . Energy in the following form:

1

( , , ) ( ) ( ( ))

N

i i

i

c b e x M x dx

  

 

(5)

ore xi( ) is the function defined by

( ) ( ) ( ) ( ) 2

i i

e x

K yx I xb y c dy(6)

Two-level formulation, starting from energy (5) which is defined as:

( , , ) ( , , ) ( ) p( )

Fc b   c bvL R  (7) withL( ) and Rp( ) being the regularization terms as defined below. The energy terms related to both are defined by

( ) ( )

L  

Hdx

( ) ( )

Rp  

p dx (8)

Which calculates the arc length of the zero-level contour ofand therefore serves to smooth the outline by penalizing its arc length [4], [10]. The potential function pDefined by

( ) (0.5) *( 1)2

p ss (9)”

(6)

5. Experimental Results and Discussion 5.1. Data set

The suggested methods viz., R-SegNet and R2-SegNet are tested over the cardiac MRI real data set and is collected from the competition of ACDC [25]. However, the study is mainly focused on the myocardial dataset of the ACDC competition and considered the following data types left ventricle, right ventricle and myocardium. The complete database collects data from a total of 100 patients as a 3D MRI images in clinical routine. Pretreatment.Later, the 3D MRI corresponds to the ACDC dataset is converted into 2D images and derived 1700 2D images as final output after applying processing mechanism. Finally the dimension of each image is to be considered as 128 × 128. The complete data is divided into three data sets training, test and validation. The size of three data sets includes 1020 MRIs, 340 MRIs and 340 MRIs.

5.2. Experiment Measures

To test the evaluation of the segmentation accuracy related to proposed method over standard methods of CNN used totally four types of the measures which are to be considered as dice similarity coefficient (DSC), the area under the curve (AUC), the Jaccard similarity (JSC) and F1-score. DSC is a measure which is used to estimate the difference of results metric among the segmentation and ground truth images respectively. The measure of DSC can be defined as follows:

2

2 2

2 T

T

DSC G

G

 

  (10)

From the (10) , PG is to be considered as product from the element-by-element among the two elements one is prediction (P) and other is ground truth (GT), and then with the

X 2to be the standard of X .Next, measure is AUC and it to be result as a probability value. The method which produces maximum of this AUS to be better compared to other methods. The AUS can be calculated using following equation:

0 0

0

0 1

( 1)

2 S n n

AUC n n

 

 (11)

Where n0 to be set of the pixels corresponds to the ground truth and n1 is number of pixels of the predicted. From (11), which is defined asS0

in r01i , here ri is the rank from the prediction model in the CMR image. The F1-score, average score among the precision and recall and it is derived best value once the precision and recall to be best and it is zero other case.

1score 2 R

F   R

  (13)

From (13), where  is precision and R R is recall. The measure which is used to improve the sample set differences and similarities as single value i.e., JSC. The maximum value of the JSC indicates the maximum similarity among the data. The measure of JSC can be defined as:

(7)

G P

P G

JSC G P

 

 

 (14)

where P is represented as prediction probability and ground truth be specified as G . 5.3. Results

The proposed methods R-SegNet and R2-SegNet with Level Set were compared over standard segmentation methods namely U-NET, ResNet, RecNet and SegNet.The complete segmentation results of each individual networks include standard and proposed as exhibited in Table 1 that shows the results of the suggested method bettering cardiac MRI segmentation that used deep learning architectures. The cross validation results over training test, proposed method produced an average of DSC value i.e., 0.9483, AUC is 0.9528, F1 score of 0.9136 and JSC is 0.8571. All benchmarks show highervalues in proposed methods compared to the standard segmentation methods. Moreover, it is observed that from the U-Net and SegNet , the DSC value of proposed method is much better, an increase of 8%. Also it is noticed that all the other benchmarks values of AUC, JSC and F1-score have much higher value than SegNet and U-Net. Therefore, in relation to the two standard methods U-Net and SegNet, the performance of proposed method is much superior. In addition, lower values of dice score is produced by the proposed model compare to standard methods and complete results of these values shown in Table1.

Table 1: Segmentation performance using various deep neural networks

From the results it shows that proposed method give better segmentation accuracy compared to classical segmentation methods like U-Net, ResNet and SegNet especially in myocardial segmentation tasks.

(8)

In addition, also performed evaluation on the ACDC data set related to both left ventricle as well as right ventricle dataset. Table 2 exhibits the cardiac segmentation results that used deep learning architectures related to left ventricle data sets, thus showing the suggested methods performing better than standard methods. Overall, the suggested method got best Dice score 0.9483 compared to SegNet with 0.7585. Similarly, accuracy of 0.9739, F1 Score 0.9428, and JSC 0.8572 all these are better compared to all standard methods.

Fig 3:Segmentation effect in the right ventricle data set.(a) Initial contour (b) Result of U-Net (c) Result of SegNet (d) Result of R-SegNet (e) Result of R2-SegNet

Also, in this work, right ventricle data set is evaluated with results shown in Table 3.

However, to test performance of proposed over standard with benchmarks in the table, the best value of dice score of 0.9572 in the right ventricle segmentation has better value over any other segmentation method. The visual perception related to right ventricle segmentation using all deep learning approaches are shown in figure3. Also observed that proposed method produced closer results ground truth image compared to other methods.

6. Conclusion

The present work proposed a set of hybriddeep learning models with level set approach for cardiac MRI image segmentation. The work mainly includes two tasks: one is segmentation using hybrid networks called R-SegNet and R2-SegNet and Region or contours extraction using Level Set method. An encoder-decoder module is presented as the segmentation module that makes a direct prediction of a segmented image when an image input is provided. A comparative evaluation with SegNet, ResNet and U-Net was attempted here to prove the validity of the proposed method. As shown by the experimental results, the proposed network exhibits more accurate and quick segmentationcapacity inthe myocardial region of the heart. Our network segmentation performs better and in a robust manner over

(9)

the other three classical segmentation networks. It also can suitably perform segmentation tasks in other medical images. In future, we will extend the function of this network to cover many more medical image segmentation tasks.

References:

1. DinthisrangDaimary, MayurBhargab Bora, KhwairakpamAmitab, DebdattaKandar.

"Brain Tumor Segmentation from MRI Images using Hybrid Convolutional Neural Networks", Procedia Computer Science, 2020.

2. Chao Cong, Hongmin Zhang. "Invert-U-Net DNN segmentation model for MRI cardiac left ventricle segmentation", The Journal of Engineering, 2018

3. Mikkili Dileep Kumar, KV Ramana.” Cardiovascular disease prognosis and severity analysis using hybrid heuristic methods”, Multimedia Tools and Applications,2020.

4. Chao Luo, Canghong Shi, Xian Zhang, Jing Peng, Xiaojie Li, Yucheng Chen. AMCNet:

5. Attention-Based Multiscale Convolutional Network for DCM MRI Segmentation",2019 6. IEEE 43rd Annual Computer Software and Applications Conference (COMPSAC), 2019 7. Mikkili Dileep Kumar,KV Ramana.” Left Ventricle of Cardiovascular Image Segmentation Using T-Segnet Hybrid and Extended Buffalo Optimization”, European Journal of Molecular & Clinical Medicine,2020.

8. Artificial Intelligence and Security", Springer Science and Business Media LLC, 2019.

9. "Information Processing in Medical Imaging",Springer Science and Business Media LLC,2017.

10. Wang, L. "Active contours driven by local and global intensity fitting energy with application to brain MR image segmentation", Computerized Medical Imaging and Graphics, 2010.

11. Diana Rojas-Ordus, Luis Jimenez-Angeles,Salvador Hernandez-Sandoval, Raquel Valdes-Cristerna. "Factor analysis of ventricular contraction using SPECT-ERNA images", 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology, 2010.

12. ChunmingLi,Rui Huang, Zhaohua Ding, J C Gatenby, D N Metaxas, and J C Gore. "A Level Set Method for Image Segmentation in the Presence of Intensity Inhomogeneities With Application to MRI", IEEE Transactions on Image Processing, 2011.

13. Chao Luo, Canghong Shi, Xiaoji Li, DongruiGao. "Cardiac MR segmentation based onsequence propagation by deep learning", PLOS ONE, 2020.

14. Chao Luo, Canghong Shi, Xian Zhang, Jing Peng, Xiaojie Li, Yucheng Chen.

"AMCNet: Attention-Based Multi scale Convolutional Network for DCM MRI Segmentation", 2019 IEEE 43rd Annual Computer Software and Applications Conference (COMPSAC), 2019.

15. Yan-Jie Zhou, Xiao-Liang Xie, Xiao-Hu Zhou,Shi-Qi Liu, Gui-Bin Bian, Zeng- GuangHou."Pyramid attention recurrent networks for realtimeguidewire segmentation and tracking in intraoperative X-ray fluoroscopy", Computerized Medical Imaging and Graphics, 2020.

16. DimahDera, NidhalBouaynaya, Hassan M.Fathallah-Shaykh. "Automated Robust Image Segmentation: Level Set Method Using Nonnegative Matrix Factorization with Application to Brain MRI", Bulletin of Mathematical Biology, 2016.

(10)

17. Wei Wang, Zhenkuan Pan, Qian Dong, Guodong Wang. "Multiphase Segmentation on CT Liver Image Using Split-Augmented Lagrangian Projection Method", 2013 Seventh International Conference on Image and Graphics, 2013.

18. Linna Yang, Dan Xin, LeyuZhai, Fang Yuan,Xiaopeng Li. "Active Contours Driven by Visual Saliency Fitting Energy for Image Segmentation in SAR Images", 2019 IEEE 4th International Conference on Cloud Computing and Big Data Analysis (ICCCBDA), 2019.

19. "Machine Learning in Medical Imaging",Springer Science and Business Media LLC,2019.

Referințe

DOCUMENTE SIMILARE

Keywords: Particle Swarm Optimization (PSO), Convolutional Neural Networks (CNN), Image classification, Pooling

Multi-source social media data sentiment analysis using bidirectional recurrent convolutional neural networks. Deep learning for automated sentiment analysis of social

Transfer learning on a collection of 2000 radiograms square measure usually accustomed train four modern convolutional neural networks, at the side of

Based on the functions of convolutional neural network layers, thermal images both normal and abnormal are taken as input image for segmentation of cancer

Abstract:A CAD (computer-aided diagnosis) framework based on a Deep Convolutional Neural Network was built in this paper.Initially, we applied Gaussian Mixture Convolutional

This project built with the assistance of very simple and basic Convolutional Neural Network(CNN) model using TensorFlow with Keras library and OpenCV to detect whether

An Accurate Prediction of Disease Using Trapezoidal Long Short Term Recurrent Neural Network Model in Big Data.. K.Tamilselvi 1

Through pre- processing, image fusion and initial tumor strip classification, tumor segmentation based on the final hybrid intelligent fuzzy Hopfield neural network algorithm,