Content based Image Retrieval Method using Fuzzy Heuristics
P.E. Rubini1, Dr.T.Guna Sekar2,A.Selvi3,V.Subashini4,Dr.R.Nagarajan5, Dr.P.Jenopaul6
1Assistant Professor, DepartmentofComputerScience and Engineering,CMR Institute of Technology, Bangalore,India. Email Id:[email protected]
2Associate Professor, Department of Electrical and Electronics Engineering, Kongu Engineering College,
Erode, Tamilnadu. Email Id:[email protected]
3Assistant Professor, DepartmentofComputerScience and Engineering, M.Kumarasamy College of Engineering,
Karur.Email Id: [email protected]
4Assistant Professor, Department of Electronics and Communication Engineering, Rajalakshmi Institute of Technology,
Chennai. Email Id: [email protected]
5Professor, Department of Electrical and Electronics Engineering, Gnanamani College of Technology, Namakkal-637018.Email Id: [email protected]
6Professor, Department of Electrical and Electronics Engineering, AdiShankara Institute of Engineering and Technology, Kerela-683574.Email Id: [email protected]
ABSTRACT
Content based image retrieval (CBIR) refers to image content that is retrieved directly, by which the images with featuresor containing certain contents will be searched in an image database. The main idea of CBIR is to analyze imageinformation by low level features of an image, which includes colour, texture, shape and space relationship of objects etc.,and to set up feature vectors of an image as its index. A new CBIR search engine is proposed using three features andsimilarity is measured and controlled by fuzzy heuristics. CBIR Search Engine relies on the characterization of primitivefeaturessuchascolour,shapeandtexturethatareautomaticallyextractedfromtheimages.Thereare severaltechniquesto deal with CBIR problems for retrieving the relevant images. CBIR proposed by using three methods. Colour feature isextracted by using histogram-based method, texture feature is extracted by using Gabor filter and shape feature is bymoment invariant algorithm. For searching the similar images with the database similarity measure is calculated and iscontrolled by using fuzzy.
Fuzzy similarity measure is implemented by using Mamdani fuzzy inference method. The useof these three algorithms ensures that the image retrieval approach produces images which are relevant to the content ofanimagequery.
Keywords-Content-based,FuzzyHeuristics,ImageRetrieval,SearchEngine.
1. INTRODUCTION
Nowadays,withlargenumberofdigitalimagesavailable on Internet, efficient indexing and searchingbecomesimportantforlargeimagestorage.Intraditional approach labeling of images with keywords,provides the diversity and ambiguity of image contents.So,content- basedimageretrieval(CBIR)approachindexesimagesbylow-levelvisualfeaturessuchascolor,texture
andshape.A typical CBIR system consists of two main parts:
(i)featureextractionand(ii)similaritymeasurement.First, features such as shape, textureand color, which constitute the image signature, aregenerated to represent the content of a given image. Thesimilarity of a query image to the images in database isthen measured using an appropriate distance
metric.Intypical content-based image retrieval approach, a usersubmits an image-based query which is then used by thesystemtoextractvisualfeaturesfromimages [1], [2].Thevisual feature is based on the
type of image
retrieval.Thesefeaturesareexaminedinordertosearchandretrievesimilarimagesfromimagedatabase.Thesi milarity of visual features between query image andimageinadatabaseiscalculatedby applyingfuzzyrules[3], [4].
In content-based image retrieval systems, a desirableimage is retrieved, from the large collection of imagesstoredintheimagedatabase,basedontheirvisualcontent.Thevisualcontentofanimageisrepresented by common attributes which are called features. Theyinclude ‘shape of the image’, ‘colour histogram of
theimage’and‘textureoftheimage’.Colourfeatureisthemostcommonlyusedvisualfeatureforimageretriev al.Manycolourmodelsareavailable that can be used to represent images such asHSI,HSV,LAB,LUVandYCrCb.Colorsplayamajor role in human perception. The most commonlyusedcolourmodelisred green blue (RGB),whereeachcomponentrepresentscolour, red,greenand blue [5], [6].
Texture isanother importantfeature of an
imagethatcanbeextractedforthepurposeofimageretrieval.Imagetexturereferstosurfacepatternswhichsho wgranular details of an image. It also gives informationabout the arrangement of different colors.
There existtwo main approaches for texture analysis. They
includestructuralandstatisticalapproaches.Instructuraltextureapproach,thesurfacepatternisrepeated.
Instatistical texture; the surface pattern is not
regularlyrepeatedinthesamepatternsuchasdifferentflower objects in a picture. Co-occurrence matrix is
a popularrepresentationoftexturefeatureofanimage [7],
[8].Itisconstructedbasedontheorientationanddistancebetween image pixels. The wavelet transform isusedforimageclassificationbasedonmulti-resolutiondecomposition of images. Among the different wavelettransform filters, Gabor filters were found to be veryeffectiveintexture analysis.
Shapefeatureplaysavitalroleinobjectdetectionandrecognition.Objectshapefeaturesproviderobustandeffi
cient information of objects in order to identify
andrecognizeimage.Shapefeaturesareimportantindescribing and differentiating the objects in an image.Shape features can be extracted from an image by usingtwo kinds of methods: contour and regions.Contourbasedmethodsarenormallyusedtoextracttheboundaryfeaturesofanobjectshape.Region- basedmethodsthatrelyonshapedescriptorsarenormallyabletoextractbothkinds offeatures: boundary andregion. Region-based methods normally use a moment-based theory such as Hu moments, Legendre momentsand Zernikemoments [9].
2. SYSTEMARCHITECTURE
The objective of using three algorithms is to develop
anintegratedimageretrievalapproachcapableofproducingefficientresults.Fig.1 showsatypicalcontent- basedimage retrieval system. Forbetterresults,theapproach ensures that the retrieved images are highlyrelevanttothequery
image.Whenauserinputsanimagequery,theimageretrievalapproachextractsfeatures based on colour, shape and texture by applyingrelevant algorithms. The extracted features are stored ina feature vector.
Then a similarity measure based onEuclidean distance anda setof fuzzy rules are appliedto produceresultsrelevantto theimagequery. A content-based image retrieval approach is based oncolour,textureandshapefeaturesandcontrolledbyfuzzy heuristics and the architecture is shown in Fig.2.
Fig.1Typicalcontent-basedimage retrieval system
Fig.2Proposed ContentBasedimageretrievalarchitecture
2.1 Histogrambasedmethod
Forthecolourfeature,weintegratetwotypesofhistogram-basedmethodsusingacolourimagehistogram and an intensity imagehistogram.For thecolour image the RGB colour model that is based on theRed, Green and Blue components. An image histogramcanbe generatedasfollows in eqn. (1)
(1) whereδb (i, j) = 1 if the v at pixel location [i, j] falls inb,andδb(i,j)=0otherwise.Similaritiesbetweendifferent histograms can be calculated using differentmethodssuchasEuclideandistanceandhistogramintersection as a similarity measure. Every pixel in animageisbasicallyrepresentedasapointinthecolourmodel such as RGB. This colour point is represented bythree values that hold the information of colour.
Theimageisrepresentedbyitshistogram.Thecolourhistogramhelpstofindtheimageswhichcontainsimilar
colour distribution. It is achieved by
measuringthesimilaritiesthroughcomputingdistancebetweentwohistograms.
2.2 GaborWaveletmethod
Thesecondelementisthetexturefeature.Forthispurpose, theGaborwavelet algorithmis used.Thewavelettransformationprovidesamulti-scaledecomposition of an imagedata. The Gabor filter isnormally used to capture energy at a certain scale and ata certain orientation. Scale and orientation are two mostimportant and useful features that are used for textureanalysis. The Gabor filter is also known as scale androtation invariantA 2D Gabor function consists of asinusoidalplanewaveofsomeorientationandfrequency,modulatedby a2D Gaussian. The Gaborfilterinspatial domainwith ‘x’ and ‘y’ value can be represented in the following eqn. (2)
where,
(2) wavelengthofcosinefactorisrepresentedbyλ;θrepresentstheorientationofthenormaltoparallelstripes of a Gabor functionin the degree; the phaseoffset in degree is represented by Ψ; the spatial aspectratiowhich specifies the elliptically of the support ofthe Gabor function is represented by γ; and
σ is thestandard deviation of the Gaussian that determines
thelinearsizeofthereceptivefield.Whenanimageisprocessed by Gabor filter; the output is the convolutionof the image I (x, y) with the Gabor function g (x, y)whichis shown in eqn. (3)
r (x,y) =l(x, y) *g(x, y) (3)
where∗ represents the 2D convolution. The process
canbeperformedatvariousorientationandscale;andpreparedfilterbank.Togeneratethefilterbank,different scale and orientation parameters help to covertheentirespatialfrequencyspacetocapturemostlytexture information with filter design. After applyingGabor filters on the image by orientation and scale, toobtainanarrayofmagnitudes which is denoted in the eqn. (4)
(4)
m=0,1, ...M-1;n=0,1, N-1
Themagnitudesrepresenttheenergycontentatdifferentorientationandscaleofimage.Themainpurpose of texture-based retrieval is to find images orregions with similar texture. The following mean μmnand
standard deviationσmn of the magnitude of
thetransformedcoefficientsareusedtorepresentthetexturefeature oftheregion and is calculated as shown in eqn. (5) and (6)
(5)
(6)
whereMrepresentsthescaleandNrepresentstheorientation.Thefeaturevectorthatrepresentsthetexture
features is created using mean μmnand standard
deviationσmnasfeaturecomponentsandthesecomponents are saved into two feature vectors and thenthese two vectors are combined in order to make thesingle feature vector that will be treated as an imagetexturedescriptor.
2.3 MomentInvariantMethod
Thethirdmainelementisshapefeature.Inthisapproach Hu moment invariant algorithm is used. TheHu
moment invariants algorithm is known as one of
themostsuccessfultechniquesforextractingimagefeaturesforobjectrecognitionapplication.The2Dmome nt of order (p +q) of a digital image f (x, y) isdefined as in the eqn. (7)
(7) for p, q = 0, 1, 2 where the summations are over thevalues of the spatial coordinates x and y spanning theimage.Thecorrespondingcentral momentiscalculated as per the eqn. (8)
(8)
where,
Here in the eqn. (8), x and y are called the Centre of the region. Hence thecentral momentsoforderup to3 canbecomputedas in the below eqn. (9)
(9) Thenormalizedcentralmomentoforder(p+q)iscalculated as shown in eqn. (10)
(10)
From Φ1 to Φ6 moments are scaling, rotation and translationinvariants and the φ7 moment is skew invariant whichenables it to differentiate the mirror images.From Φ1 to Φ7momentsareusedto calculate thefeaturevectors and the formula to calculate is shown in eqn. (11)
(11)
This set of normalized central moment is invariant totranslation,rotationand scalechangesinanimage.
3.4.EuclideanDistanceMethod
Similarity between two images is measured numericallythat reflects the strength of connections between them.Euclidean distance is used to calculate the similaritybetweentwofeature vectorsand is computed in eqn. (12)
(12)
Where Mk and Mt are image query and image
databaserespectively,iisafeaturerange.Closerdistancerepresentsthehigher similaritybetweenimages.
3.5Fuzzysimilaritymeasure
Fig.3 Fuzzy heuristics
Fuzzy heuristics is used to measure similarity betweenthe query image and the database images in order toretrieveanddisplay relevantorsimilarresultstothe user query and is represented in Fig.
(3).Threetypesofpreferencesaretaken;thefirst priority is given to the shape features, as it is notaffectedbyexternalfactors,andinvarianttotherotation, translation and orientation. The second priorityis given to the colour features, as it is invariant to
therotationandtranslation.Thethirdpriorityisgiventothetexturefeatures.TheMamdanifuzzyinferencemet hod is used to perform fuzzy rules. After obtainingthe relevant images to the query image, the commonimagesbetweenX,YandZsetofimagescanberetrieved. The common set of images is considered themost relevant images. Commonality is measured usingthebelowcriteria.
X = Shape features are used to calculate the distancebetweenqueryanddatabase image Y = Colour features are used to calculate the distancebetweenqueryanddatabase image.
Z = Texture features are used to calculate the distancebetweenqueryanddatabase image S = Imagesimilarity
By adopting the steps, a set of fuzzy rules to process
theresultsachievedbyapplyingthethreedistinctalgorithms.
Step 1: Define a number of inputs. In this case threeinputs are used such as shape distance, colour distanceand texture distance between query image and databaseimages.
Step 2: The membership functions for three types ofinput have been defined. There are three different typesof fuzzy set that identified each input as low, mediumand high.
Step 3: Three types of output fuzzy sets have beendeclared such as high similar, medium similar and lowsimilar.
Step 4: A fuzzy rule can be defined as a conditionalstatementsuchasif then.Fuzzy rulesappliedusinglogicaloperator.
Step5:ToprocesstheMamdanifuzzyinferencemethod the crisp inputs are taken and fuzzifier todetermine the degree to
Which these inputs belong toeachofthe appropriate fuzzyset.
Step 6:Apply theAND fuzzy operator to get onenumber that represents the result of antecedent of rules.Theoutputisasingle truthvalue.
Step 7: The process of unification of the output of allthe rules that have been used until last step. The outputof this stepis onefuzzy setforeach outputvariable.Theprocessiscalledanaggregation.
Step 8: Lastly the aggregate output fuzzy set shouldtransform toasinglecrispnumber.Then process ofdefuzzificationistodone it.
3. EXPERIMENTALRESULTS
Thefeaturevectorforthe following images is calculated and shown in Table 1.
Table 1: Image and the respective feature vector
Thesimilaritybetweentwoimages (i)and (ii) iscalculatedbyusingEuclideandistance and is represented in the following Table 2.
Table 2 Distance calculation between two images.
Images Similarity
Measure
(a) (b)
Distance=0.01876.
Theprecisionandrecallarecalculatedfortheevaluationofthe resultbased onthefollowing eqn. (13) Precision=I/V
Recall= I/R (13) Iisnumberofimagesretrievedthataresimilartothequeryimage
V is total number of images retrieved
Risthetotalnumberofimagesinthe databasethataresimilartothe queryimage The following table 3 provides the list of images retrieved by the model upon querying with the input image.
Table 3 List of retrieved images
Image Feature Vector
Colour Feature:
TheRGBvalue fortheimage is:
Red: 115.29 Green:86.36 Blue: 63.50
Texture Feature:
Mean: 20.0635
Standarddeviation:21.98975 Shape Feature
Table 4 Precisionandrecallvaluesforvariousimages Sl. No. Images Precision Recall
1 Image1 0.5 0.3
2 Image2 0.7 0.4
3 Image3 0.6 0.33
4 Image4 0.53 0.3
5 Image5 0.60 0.34
6 Image6 0.85 0.48
7 Image7 0.67 0.38
8 Image8 0.9 0.52
9 Image9 0.35 0.2
Theprecisionandrecallvalue forthequeryimageand the retrieved images is calculated as per the eqn.
(13) and tabulated in the following Table 4 and also plotted in the graph shown in Fig.4
Fig.4Precisionandrecallontheobtainedresult 4. CONCLUSION
Imageretrievalapproachwhichisbasedoncolour,textureandshapefeatures,controlledbyfuzzyheuristics is used. The approach is based on the threewell known algorithms: colour histogram, texture andmoment invariants. The use of these three algorithmsensuresthattheimageretrievalapproachproduce results which are highly relevant to the content of queryimage, by taking into account the three distinct featuresof the image and similarity metrics based on Euclideanmeasure. The colour histogram is used to extract thecolour features of an image using four components suchas Red, Green, Blue and Intensity. The Gabor filter isused to extract the texture features and the Hu
0 0.2 0.4 0.6 0.8 1
Image 1
Image 2
Image 3
Image 4
Image 5
Image 6
Image 7
Image 8
Image 9
Chart Title
Precision Recall
momentinvariantisusedtoextracttheshapefeaturesofanimage. The evaluation is carried out using the standardPrecision and Recall measures, and the results obtainedare compared with the image query.
The work based onspace relationship will be further analyzed in the futureenhancement.
REFERENCE
[1] K. Iqbal, M.O. Odetayo andA. James,Content-basedimageretrievalapproachforbiometricsecurity using colour, texture and shape featurescontrolled by fuzzy heuristics, Journal of Computerand System Sciences,78,2012,1258–1277
[2] A.K.Jain,A.RossandS.Prabhakar,Anintroduction to biometric recognition, IEEE Trans.CircuitsSyst. Video Technol.,14 (1),2004,4–20.
[3] T. Chaira and A.K. Ray, Fuzzy measures for colorimage retrieval, Fuzzy Sets and Systems, 150, 2005,545–560.
[4] K.EricssonandA.Lehmann,Expertandexceptionalperformance:Evidenceofmaximaladaptation to task constraints, Annu. Rev. Psychol.,47 (1),1996,273–305.
[5] M.Smids,Backgroundsubtractionforurbantraffic monitoring using Webcams, Master thesis,UniversiteitVan Amsterdam, FNWI,2006.
[6] S. Jeong, C.S. Won and R.M. Gray, Image retrievalusing color histograms generated by Gauss mixturevectorquantization,Comput.VisionImageUnderst.94(1–3),2004,44–66.
[7] K. Iqbal, M.O. Odetayo and A. James, Integratedimage enhancement method for biometric
security,Proc. The 2011 IEEE International Conference
onSystems,Man,andCybernetics,Anchorage,Alaska, 2011.
[8] R.Mudigoudar,S.Bagal,Z.Yue,P.Lakshmiand P. Topiwala, Video super-resolution: From Qvga toHDinreal-time,ProceedingsoftheSPIE,vol.7443,2009,74430W1–74430W12.
[9] T.AcharyaandA.K.Ray,ImageProcessingPrinciples andApplication, John Wiley &
Sons,Inc.,2005.
