Original papers
DOI: 10.11152/mu-936Is Contrast Enhanced Ultrasonography a useful tool in a beginner’s hand? How much can a Computer Assisted Diagnosis prototype help in characterizing the malignancy of focal liver lesions?
Tudor Voicu Moga
1, Alina Popescu
1, Ioan Sporea
1, Mirela Danila
1, Ciprian David
2, Vasile Gui
2, Nicoleta Iacob
3, Gratian Miclaus
3, Roxana Sirli
11Department of Gastroenterology and Hepatology, “Victor Babes” University of Medicine and Pharmacy, 2Electronics and Telecommunications Faculty, “Politehnica” University, 3Department of Anatomy and Embryology, “Victor Babes”
University of Medicine and Pharmacy, Timișoara, Romania
Received 20.12.2016 Accepted 26.03.2017 Med Ultrason
2017, Vol. 19, No 3, 252-258 Corresponding author: Alina Popescu
8 Barbu Iscovescu Street, Sc. A, Ap. 7 300561 Timisoara, Romania
Phone: 0040 746 778526, Fax: 0040 256 488003 E-mail: [email protected]
Introduction
The widespread and easy access made the ultrasound examination ubiquitous among medical doctors. Ultra- sound is the first line imaging method for detecting fo- cal liver lesions (FLLs). Contrast Enhanced Ultrasound (CEUS) has become a validated easy-to-use, minimum invasive imaging method, for the quantitative and quali-
tative evaluation of FLLs. The efficacy and the utility of this technique are emphasized in the European Federa- tion of Societies for Ultrasound in Medicine and Biology (EFSUMB) guidelines published in 2004 [1], updated in 2008 [2] and in October 2012 [3]. There are also some multicenter studies that demonstrate the accuracy of CEUS in characterizing FLLs [4,5]. Both methods are still operator-dependent and few studies have been made to overcome this drawback, having in mind the useful- ness of a fast and reliable imaging diagnosis of a FLL [6]. It is quite hard to quantify the experience you need in order to be confident in using CEUS as a first line imag- ing method after a lesion was found in B-mode.
Computer assisted diagnosis (CAD) is still one of the major research topics in radiology and medicine, Abstract
Aim: Contrast enhanced ultrasound (CEUS) improved the characterization of focal liver lesions (FLLs), but is an operator- dependent method. The goal of this paper was to test a computer assisted diagnosis (CAD) prototype and to see its benefit in assisting a beginner in the evaluation of FLLs. Material and method: Our cohort included 97 good quality CEUS videos [34% hepatocellular carcinomas (HCC), 12.3% hypervascular metastases (HiperM), 11.3% hypovascular metastases (HipoM), 24.7% hemangiomas (HMG), 17.5% focal nodular hyperplasia (FNH)] that were used to develop a CAD prototype based on an algorithm that tested a binary decision based classifier. Two young medical doctors (1 year CEUS experience), two experts and the CAD prototype, reevaluated 50 FLLs CEUS videos (diagnosis of benign vs. malignant) first blinded to clinical data, in order to evaluate the diagnostic gap beginner vs. expert. Results: The CAD classifier managed a 75.2% overall (benign vs.
malignant) correct classification rate. The overall classification rates for the evaluators, before and after clinical data were:
first beginner-78%; 94%; second beginner-82%; 96%; first expert-94%; 100%; second expert-96%; 98%. For both beginners, the malignant vs. benign diagnosis significantly improved after knowing the clinical data (p=0.005; p=0,008). The expert was better than the beginner (p=0.04) and better than the CAD (p=0.001). CAD in addition to the beginner can reach the expert diagnosis. Conclusions: The most frequent lesions misdiagnosed at CEUS were FNH and HCC. The CAD prototype is a good comparing tool for a beginner operator that can be developed to assist the diagnosis. In order to increase the classification rate, the CAD system for FLL in CEUS must integrate the clinical data.
Keywords: Contrast Enhanced Ultrasound; beginner; expert; focal liver lesions; computer assisted diagnosis
having a pattern recognition software as a main feature, that calculates the likelihood of different lesions as being malignant or benign. This application is used in diagnos- tic radiology, for different imaging methods [computer tomography (CT), magnetic resonance imaging (MRI), conventional radiology (RX) and ultrasonography (US)]
in organs such as breast, lungs, brain, liver, kidney, colon, etc. There are only few studies regarding the utility of a CAD system for FLLs diagnosis by CEUS. Sugimoto et al [7] proposed two schemes for a CAD classifier of FLLs using CEUS, with encouraging results.
VueBox, from Bracco Italy, is a quantification tool- box for CEUS that can analyze General Imaging (GI) perfusion and the liver Dynamic Vascular Pattern (DVP) which moves one step forward the CAD in CEUS con- cept. Though, in order to use such a tool perfect examina- tion conditions and good quality CEUS images and video clips are needed, thus returning to the initial operator de- pendence problem. The main purpose of a CAD system is to reduce the reading time and increase the diagnostic accuracy. We have to underline the differences in mean- ing of the words detecting/diagnosis. CAD in CEUS for FLLs will focus on the diagnosis of lesions characterized by CEUS (B-mode detection being a condition), detec- tion software for the liver would engage other resources and in the same time other difficulties. Our study is ori- ented towards the utility of CEUS for an inexperienced user and to assess which tools can be used in order to improve the diagnosis accuracy of FLLs in daily practice.
The aim of our study was to create a CAD prototype for CEUS in evaluating FLLs, that can orientate the diag- nosis to malignancy and which, together with the clinical data, can raise the diagnostic accuracy for the beginners.
Through this, we wanted to highlight the most challeng- ing lesions regarding FLLs interpretation, which a begin- ner might encounter during CEUS.
Material and method
A total of 97 CEUS video-cases of FLLs were used to develop the CAD prototype. From 97 FLLs, 33 (34%) were hepatocellular carcinomas (HCC), 12 (12.3%) hypervascular metastases (HiperM), 11 (11.3%) hypo- vascular metastases (HipoM), 24 (24.7%) hemangio- mas (HMG) and 17 (17.5%) focal nodular hyperplasia (FNH). The video-cases were chosen from the database of our department according to quality parameters (ob- vious enhancing pattern, good ultrasound examination, good acoustic window). Contrast enhanced CT, MRI, or histology were available in each case to confirm the final diagnosis. The main focus was to analyze the five most frequent types of lesions: hepatocellular carcinoma,
hypervascular metastases, hypovascular metastases, he- mangioma and focal nodular hyperplasia. From the 97 FLLs, 33 of them were on cirrhotic liver (all the HCCs), the rest being detected on a normal liver on ultrasound examination. All selected video-cases were carefully chosen to have a typical CEUS pattern as described in EFSUMB guidelines [3].
The lesions were detected and assessed by standard ultrasound (B-mode) according to their number, size and location and then evaluated by CEUS with low mechanic index, convex probe (4C1), using SonoVue® (Bracco, Italy) as contrast agent, approx. 2.4 ml/lesion, followed by a 10-mL saline bolus. All examinations were made with a Siemens Acuson S2000 ultrasound machine with CEUS software and were performed by experienced op- erators (Level II – advanced and III – expert, according to the Romanian Society of Ultrasound in Medicine and Biology classification).
Each examination respected the standard EFSUMB protocol for CEUS [3]. The enhancement pattern of each lesion was followed up in 3 phases: arterial phase 10-30 sec, portal phase 30-120 sec and late phase >120 sec. All lesions were evaluated by four video-clips: one (10 second long) of conventional ultrasound study; and the other three, CEUS phases (arterial, portal and late phase), approx. 10-30 seconds for each phase. The le- sions were characterized by comparing them with the normal enhancement of the surrounding liver, appear- ing as: homo-heterogeneous hyperenhanced – mean- ing the FLL’s enhancement pattern was homogeneous or presented irregularities; rim-like hyperenhancement – peripheral hyperenhancement, less than 1/4 of the tu- mor’s diameter; isoenhancement – similar enhancement pattern with the adjacent parenchyma at the same depth;
hypoenhancement – the FLL enhances less than the adja- cent parenchyma at the same depth; and wash-out – hy- per or iso-enhancement in the arterial phase followed by hypoenhancement in the portal or late phases. The en- hancement pattern of each lesion type, according to the EFSUMB 2012 guidelines, detailed in [3], was used as a template to characterize the FLLs.
CAD prototype – Algorithm description
Malignant/benign ruled based discrimination was ob- tained by computing the average relative intensity and by analyzing its evolution from the arterial to the late phase.
These image attributes were considered in a user selected region of interest (ROI). The ROIs had been selected by one of the experts and included the entire lesion previous- ly detected on standard ultrasound. By average relative in- tensity we understood the average color of the lesion area against the average color of the adjacent parenchyma. The region where the normal tissue was considered was taken
Fig 1. A – Average Intensity variation in the Arterial Phase, with a wash-in pattern; B – Lesion area selected in a region of interest in B-mode; C – Lesion area selected in a region of interest in CEUS.
Fig 2. A – Average Intensity variation in Late Phase, with a wash-out pattern; B – Lesion area selected in a region of inter- est in B-mode; C – Lesion area selected in a region of interest in CEUS.
adjacent to the ROI and had the same area. The equations related to the benign/malign decision are shown in table I.
In the left cell the average intensity is presented. We call relative average intensity the quantity obtained by divid- ing the average intensity of the lesion by the average in- tensity of the adjacent parenchyma (Table I- central cell).
The criteria used to decide if we have the case of ma- lignant FLL (Table I- right cell) are in concordance with the assumptions that malignant FLL presents wash out in the late phase. According to the average relative intensity (RAvgInt), the result was ≥1 in the arterial phase and ≤1 in the late phase for a malignant pattern, thus reducing the diagnosis to a binary decision (fig 1 and fig 2).
The spatial instability of the data, given by the pa- tient’s respiratory movement and by the transducer movement, was surpassed by probabilistic filtering. Our hypothesis considers that the ROI was centered in most cases on the lesion, a fact that enables us to use the tem- poral median filter. The average relative intensity was measured for each frame. That value was the value con- sidered for the final benign/malign decision.
CEUS FLLs assessment
In order to estimate or to quantify the diagnostic gap between a beginner vs. expert in establishing the diagno- sis of malignancy in a FLL, two young medical doctors under fellowship in the Gastroenterology department, with CEUS experience less than one year, and two ul- trasound experts, with more than 5 years’ experience in
CEUS, were asked to reevaluate 50 CEUS video-cases (first 10 of each lesion type) from the same database. In the first phase, both beginners and seniors were asked to differentiate malignant vs. benign lesions blinded to clin- ical data. In the second phase, if misdiagnosed (failure to differentiate between malignant vs. benign), the clinical data were revealed (FLL detected on a normal liver, a cirrhotic liver or a patient with known neoplasia) and the video-cases were reassessed. Thirdly, the CAD prototype was used to randomly classify, the previously 50 FLLs seen by the experts and beginners, into malignant or be- nign, based on an algorithm that tested a binary decision based classifier.
Statistical analysis
Data are expressed as percentages of successful di- agnosis for each examiner, before and after knowing rel- evant clinical data, and CAD, respectively. We used the Wilcoxon signed-ranks test to compare results within ex- aminers and between examiners and CAD. The statistical analysis was performed for a 95% confidence interval, using IBM SPSS Statistics Package.
Results
The first beginner misdiagnosed 4 out of 10 FNHs, 3 out of 10 HCCs, 2 out of 10 HMGs, 1 out of 10 hyper- vascular metastases and 1 out of 10 hypovascular me- tastases (fig 3). After knowing the clinical data, the first Table I. Specific formulas for the average relative intensity (from left to right): average intensity
formula; relative average intensity between lesion and parenchyma regions; malignancy criteria.
beginner misdiagnosed 2/10 FNHs and 1/10 HCCs. The second beginner misdiagnosed 3/10 FNHs; 3/10 HCCs, 2/10 HMGs and 1/10 hypervascular metastases. After revealing the clinical data, the second beginner misdi- agnosed 1/10 HCCs and 1/10 HMGs. The first expert
misdiagnosed only 1 out of 10 FNHs, 1/10 HCCs and 1/10 HMGs. After revealing clinical data all lesions were correctly diagnosed. The second expert misdiagnosed 1 out of 10 FNHs and 1out of 10 HCCs. After the clini- cal data were revealed, only 1 out of 10 FNH was still misdiagnosed.
The overall classification rates for the evaluators, be- fore and after clinical data were revealed were: first be- ginner without knowing clinical data 78 %; after know- ing clinical data 94%, second beginner without clinical data 82%; with clinical data 96%. First expert: without clinical data 94%; with clinical data 100%. Second ex- pert: without clinical data 96%; with clinical data 98%.
The CAD ruled based classifier managed an overall classification rate of 75.2%, malignant set classification rate-70.5% and benign set classification rate-80%.
All beginners’ malignant vs. benign diagnosis signifi- cantly improved after knowing the clinical data (table II).
The expert was better than the beginner and better than CAD (table III). A comparison between the performance of experts and beginners, when CAD was favorable dem- onstrated that the beginner plus CAD can reach the ex- pert diagnosis (table IV). The touchstones of the begin- ner’s evaluation were HCC and FNH.
Discussions
The need for CAD software for the liver is neces- sary if we take into consideration the incidence of focal liver lesions and the impact upon the medical healthcare system [8-12]. CEUS in liver pathology has helped the characterization of focal liver lesions a lot, increasing the specificity and giving a boost to ultrasound users re- garding the FLLs discrimination [13]. To our knowledge there is no published data regarding the learning curve in CEUS but we assume that according to EFSUMB recom- mendations [14], operators must have a competence level higher than level I according to minimal training require- ments in ultrasound (EFSUMB 2006) to perform CEUS [15]. According to our local experience, a minimum of one year is required to gain experience in CEUS, by ob- serving contrast studies done by experts.
Several multicenter studies regarding CEUS have confirmed the utility of this method: the German DEG- UM trial, with more than 1000 lesions, which showed a diagnostic accuracy of CEUS of 90.3% (in differentiating between malignant and benign lesions) [4]; the French STIC trial [5] with a diagnostic accuracy of 86% and the Romanian multicenter study [16] with a diagnostic accu- racy of 89.3%. When comparing CEUS vs. CT/MRI for the diagnosis of FLLs [17,18], there were no statistical differences. CEUS for FLLs has already proved its cost Fig 3. The result chart representing each misdiagnosed lesion
made by the beginners (without knowing the clinical data) and by the computer assisted diagnosis.
Table II. Diagnosis accuracy of contrast enhanced ultrasound in focal liver lesions after knowing the clinical data (p value of Wilcoxon test).
Examiner/
Success Blinded to
clinical data Unblinded to clinical data p*
1st Beginner 78% 94% 0.005
2nd Beginner 82% 96% 0.008
1st Expert 94% 100% 0.083
2nd Expert 96% 98% 0.564
Table III. The success rate of experts in comparison with begin- ners and computer assisted diagnosis (*p, significance level of Wilcoxon test).
Examiner Percentage of successful diagnosis (%)
Beginner
(p*) Expert
(p*) CAD
(p*)
Beginner 80 – 0.04
Expert 95 0.04 – 0.01
CAD 75.2 0.01 –
Table IV. Comparison between the performance of experts and beginners, when computer assisted diagnosis was favorable (*p value of Wilcoxon test)
Examiner (% correct diagnosis)
p value*
1st Beginner +
CAD (94%) 2nd Beginner + CAD (98%)
1st Expert (94%) 1 0.317
2nd Expert (96%) 0.655 0.564
efficiency as a first line imaging method [8,12] compared to CT or MRI, making it well worth considering in daily use. With all these features and advantages that CEUS offers, it is obvious that CAD software for CEUS still has potential that can be exploited.
In order to implement CAD for CEUS, the drawbacks of ultrasound must be overcome. First of all, a good and valid database is required so that the algorithm used for the software can assimilate the standard enhancing pat- terns of lesions. This implies a correct examination and a good acoustic window. One of the crucial parts of a CAD system is the software development. A basic knowledge in the field of engineering that a physician has, is not enough. A symbiosis between the two fields (medicine and engineering) is required, skills in image processing, data analysis and software development will be the nec- essary ingredients for reliable software. There are several examples in the literature that have proved the utility of a CAD system in ultrasound. The wide range of use has made it useful in breast lesion interpretation [19], quan- tifying liver steatosis [20] and even in assisting “killing”
the cancer cells [21].
The ideal CAD will be one that can be of a real use for the young examiners. It must bring a diagnostic ben- efit in characterizing a FLL, especially in those lesions that can be misdiagnosed (atypical lesions, deep ones, small ones etc.). The goal is to create a tool that helps the young examiners in the diagnostic decision. In order to be confident in putting a diagnosis with the help of CEUS, you need a minimum experience in CEUS and ultrasound. There is not a validated and quantifiable ul- trasound evaluation that can measure the experience you need in ultrasound/CEUS, this is why a CAD will bring a true benefit for the inexperienced users.
Unlike the study of Sugimoto et al [7], in which the authors developed two CAD schemes using CEUS, one based on physicians’ subjective classification and the other based on quantitative analysis that uses artificial neural networks and decision trees in order to differen- tiate FLLs, we focused mainly on the malignant/benign fast discrimination of a FLL. In Sugimoto et al study [7] 137 nodules were evaluated: 74 HCCs [23 well- differentiated (w-HCC), 36 moderately differentiated (m-HCC) and 15 poorly differentiated (p-HCC)], 33 liver metastasis and 30 hemangiomas, classifying them into eight patterns according to the enhancement: (1) absent; (2) dotted; (3) peripheral rim like; (4) peripheral nodular; (5) central with spoke wheel-shape; (6) diffuse heterogeneous; (7) diffuse homogeneous; and (8) oth- ers. Four artificial neuronal networks (ANN) were used afterwards as decisions models for the evaluators. The performance of the subjective classification for the 137
FLLs were 84.8% for metastasis, 93.3% for hemangio- ma, 65.2% for w-HCC, 41.7% for m-HCC, and 80.0%
for p-HCC. The average classification accuracies for FLLs type three and five were 94.2% and 71.5%, re- spectively.
The second scheme that Sugimoto used [7] is based on a micro-flow imaging technique (MFI; Toshiba Medi- cal Systems Co., Otawara, Japan) using high mechanical index disruptive flash frames [22]. Through this tech- nique essential details about the lesions’ vessels had been obtained. CEUS with MFI has helped to differentiate the HCCs’ and other FLLs’. Different image features were used as input for the CAD. They used 103 nodules, from which 61 HCCs (24 w-HCC, 28 m-HCC and 9 p-HCC), 26 liver metastases and 16 liver hemangiomas. In this second computerized scheme Sugimoto el al [7] divided the process in three steps: 1. processing the images; 2.
extraction of image features and 3. application of ANNs.
The classification accuracy of the second CAD for the 103 lesions were 88.5% for metastasis, 93.8% for he- mangioma, 79.2% for w-HCC, 50.0% for m-HCC and 77.8% for p-HCC. The classification rate for all HCCs was 86.9%. The average classification accuracies for FLLs type three and five were 88.3% and 75.7%.
Recently a new algorithm was proposed by Gastos [23] in which a quantification algorithm for the detec- tion and evaluation of FLLs with the help of CEUS was tested. The study included 52 CEUS video sequences (30 benign, 22 malignant) in which lesion contour extraction and time intensity curve (TIC) were used that could be computed into the support vector machines (SVMs) clas- sification algorithm in the design of the image analysis mode. The highest classification accuracy from the SVM model was 90.3%, with 93.1% sensitivity and 86.9%
specificity.
Perhaps, more important than the correct discrimina- tion of the FLLs is to exclude the malignancy. The malig- nant/benign status is influencing the therapeutic strategy and prognosis. In our study, after having the final results of the evaluators, we analyzed the weakest point, “Achil- les heel”, in evaluating the FLLs. We noticed that the FNH and HCC were the most frequent FLLs that were misdiagnosed by the beginners (Table 1 and 2, Fig. 8 and 9). This might be due to the fact that according to the EFSUMB 2012 guidelines [3], both lesions are hyperen- hancing in the arterial phase and FNH, as an additional feature, can be isoenhancing in the late phase like the HCC, making it worthy of expertise (Fig. 5). This prob- lem might be solved by using Sonazoid (perfluorobutane;
GE Healthcare, Oslo, Norway), as a contrast agent which allows a better late phase and parenchymal observation [24]. Both beginners increased significantly the diagnos-
tic accuracy for the HCC if the clinical data revealed a cirrhotic liver; thus, we can assert that the presence of cirrhosis influenced the final diagnosis.
Our CAD software had some good results on the ma- lignant/benign fast discrimination but cannot compare with the expert results. Even though we used the same database to develop and test the CAD, if we take a close look at the lesions we notice that some of the misdiag- nosed lesions by the beginners were correctly diagnosed by the CAD (Fig. 9); To summarize the algorithm that we propose, if a young investigator has doubts when eval- uating a lesion through CEUS imaging, he can use the CAD software to orientate the diagnostic, thus achiev- ing better results. We noticed that the clinical data had a major impact on the beginner’s results (being the major criterion of the results’ improvement), underlying an ob- vious fact that the imaging investigation should be made by clinicians and not by technicians. The experts’ results were not significantly influenced after knowing the clini- cal data, maybe due to the fact that the CEUS movies might have been previously seen having in mind the ret- rospective approach.
In order to increase the classification rate, in future developments, we envisage introducing as many features as possible, like tracking algorithms for the ROI, more suitable probabilistic filters and a number of texture fea- tures that will increase the method’s robustness. Clinical data must be integrated into the software as it helps the diagnostic accuracy. Our prototype is more an assisting tool than a complex post processing one. However, the lack of research in this field makes this concept a good track to start. CAD for the liver is still at the beginning, a number of issues are to be improved and new, faster algorithms should be developed.
An important bias of our study is the fact the some of the lesions might have been previously seen by the experts during the last years and the beginners did not perform the CEUS investigations, they just reevaluated the video clips from the data base. Another limitation of the study is the lack of integration of clinical data into the CAD algorithm. We can mention the fact that all CEUS video clips were of good quality but even so the success rate of our CAD was less than the results of the beginners.
The performance of CAD in the daily routine must be evaluated; however, improvements must be done in order to achieve a reliable software. Through this software we attempted the objective of developing a CAD for FLLs just by reducing the CEUS evaluation to a pattern hyper- enhancement or hypoenhancment decision, which might be a very easy task in “good quality videos“. Achieving this necessitates much more sophisticated software and much more hardware capacity.
Conclusions
The most frequent lesions misdiagnosed at CEUS were FNH and HCC. The CAD prototype is a good com- paring tool for a beginner operator that can be developed to assist the diagnosis. In order to increase the classifica- tion rate, the CAD system for FLL in CEUS must inte- grate the clinical data.
Acknowledgement: The research published in this pa- per was made with support from the grant INOV-IAHEP awarded by the “Victor Babes” University of Medicine and Pharmacy Timisoara, in PROGRAMUL III – C2 – PCFI – 2015/2016.
Conflict of interests: none
References
1. Albrecht T, Blomley M, Bolondi L, et al. Guidelines for the use of contrast agents in ultrasound. Ultraschall Med 2004;25:249-256.
2. Claudon M, Cosgrove D, Albrecht T, et al. Guidelines and good clinical practice recommendations for contrast-en- hanced ultrasound (CEUS) – update 2008. Ultraschall Med 2008;29:28-44.
3. Claudon M, Dietrich CF, Choi BI, et al. Guidelines and good clinical practice recommendations for Contrast En- hanced Ultrasound (CEUS) in the liver – update 2012: A WFUMB-EFSUMB initiative in cooperation with repre- sentatives of AFSUMB, AIUM, ASUM, FLAUS and ICUS.
Ultrasound Med Biol 2013;39:187-210.
4. Strobel D, Seitz K, Blank W, et al. Contrast-enhanced ul- trasound for the characterization of focal liver lesions-di- agnostic accuracy in clinical practice (DEGUM multicenter trial). Ultraschall Med 2008;29:499-505.
5. Tranquart F, Le Gouge A, Correas JM, et al. Role of con- trast-enhanced ultrasound in the blinded assessment of fo- cal lesions in comparison with MDCT and CEMRI: Results from a multicenter clinical trial. EJC 2008;Supplements 6:9-15.
6. Sporea I, Sirli R. Is Contrast Enhanced Ultrasound (CEUS) ready for use in daily practice for evaluation of Focal Liver Lesions? Med Ultrason 2014;16:37-40.
7. Sugimoto K, Shiraishi J, Moriyasu F, Doi K. Computer-aid- ed diagnosis for contrast-enhanced ultrasound in the liver.
World J Radiol 2010;28:215-223.
8. Sirli R, Sporea I, Martie A, Popescu A, Dănilă M. Contrast enhanced ultrasound in focal liver lesions-a cost efficiency sudy. Med Ultrason 2010;12:280-285.
9. Jang HJ, Yu H, Kim TK. Imaging of focal liver lesions.
Semin Roentgenol 2009;44:266-282.
10. Layer G, Delorme S. Radiological diagnosis of hepatic tu- mors. Part II: Identification and differential diagnosis. Ra- diologe 2007;47:919-931.
11. Giesel FL, Delorme S, Sibbel R, Kauczor HU, Krix M.
Contrast-enhanced ultrasound for the characterization of incidental liver lesions – an economical evaluation in com- parison with multi-phase computed tomography. Ultra- schall Med 2009;30:259-268.
12. Westwood M, Joore M, Grutters J, et al. Contrast-enhanced ultrasound using SonoVue® (sulphur hexafluoride mi- crobubbles) compared with contrast-enhanced computed tomography and contrast-enhanced magnetic resonance imaging for the characterisation of focal liver lesions and detection of liver metastases: a systematic review and cost- effectiveness analysis. Health Technol Assess 2013;17:1- 13. Matsui O. Detection and characterization of hepatocel-243.
lular carcinoma by imaging. Clin Gastroenterol Hepatol 2005;3:S136-S140.
14. Minimum training requirements for the practice of Medical Ultrasound in Europe. Ultraschall Med 2010;31:426-427.
15. Education and Practical Standards Committee, European Federation of Societies for Ultrasound in Medicine and Bi- ology. Minimum training recommendations for the practice of medical ultrasound. Ultraschall Med 2006;27:79-105.
16. Sporea I, Badea R, Popescu A, et al. Contrast-enhanced ul- trasound (CEUS) for the evaluation of focal liver lesions – a prospective multicenter study of its usefulness in clinical practice. Ultraschall Med 2014;35:259-266.
17. Seitz K, Strobel D, Bernatik T, et al. Contrast-Enhanced Ultrasound (CEUS) for the characterization of focal liver lesions-prospective comparison in clinical practice: CEUS vs. CT (DEGUM multicenter trial). Parts of this manuscript
were presented at the Ultrasound Dreiländertreffen 2008, Davos. Ultraschall Med 2009;30:383-389.
18. Seitz K, Bernatik T, Strobel D, et al. Contrast-enhanced ultrasound (CEUS) for the characterization of focal liver lesions in clinical practice (DEGUM Multicenter Trial):
CEUS vs. MRI–a prospective comparison in 269 patients.
Ultraschall Med 2010;31:492-499.
19. Gu P, Lee WM, Roubidoux MA, Yuan J, Wang X, Carson PL.
Automated 3D ultrasound image segmentation to aid breast cancer image interpretation. Ultrasonics 2016;65:51-58.
20. Mihăilescu DM, Gui V, Toma CI, Popescu A, Sporea I.
Computer aided diagnosis method for steatosis rating in ultrasound images using random forests. Med Ultrason 2013;15:184-190.
21. Kosheleva OK, Lai TC, Chen NG, Hsiao M, Chen CH. Se- lective killing of cancer cells by nanoparticle-assisted ultra- sound. J Nanobiotechnology 2016;14:46.
22. Shiraishi J, Sugimoto K, Moriyasu F, Kamiyama N, Doi K. Computer-aided diagnosis for the classifcation of focal liver lesions by use of contrast-enhanced ultrasonography.
Med Phys 2008;35:1734-1746.
23. Gatos I, Tsantis S, Spiliopoulos S, et al. A new automated quantification algorithm for the detection and evaluation of focal liver lesions with contrast-enhanced ultrasound. Med Phys 2015;42:3948-3959.
24. Luo W, Numata K, Kondo M, et al. Sonazoid-enhanced ultrasonography for evaluation of the enhancement pat- terns of focal liver tumors in the late phase by intermittent imaging with a high mechanical index. J Ultrasound Med 2009;28:439-448.
