View of Detection of Sardine Fish Freshness Using Deep Convolution Neural Network

(1)

Annals of R.S.C.B., ISSN:1583-6258, Vol. 25, Issue 4, 2021, Pages. 16063 - 16070 Received 05 March 2021; Accepted 01 April 2021.

Detection of Sardine Fish Freshness Using Deep Convolution Neural Network

A.C.Kaladevi

¹

,RamyaPerumal

²

,K.Anantha Priya

³

1Professor, Dept. of CSE, Sona College of Technology, Salem

1[email protected]

2Assistant Professor, Dept. of CSE, Sona College of Technology, Salem

2[email protected]

3Final year, Dept. of CSE,Sona College of Technology,Salem [email protected]

ABSTRACT

Fresh Fish food processing is the most important factorin the food production industry. Fish freshness is the main indicator for assessing the quality of fish which is essential for commercial purposes. Mostly the consumerswish to buy and eat freshly caught fish but in reality the local markets showcase and sell two-week-older fishes thatare collected from the seas. To concern the health offish consumers, one must ensure the quality of the fish before consuming it. Therefore, we have used a deep learning technique to increase the accuracy in detecting the freshness of the sardine fish. In our proposed system, Deep Convolution Neural Network is used to extract the features and detectthe qualityof the fish. On experimental analysis, we obtained a remarkable result in terms of accuracy, sensitivity, specificity, Positive Predictive Value(PPV),Negative Predictive Value(NPV) and f1-score.

KEYWORDS Image Processing, Deep Learning, CNN.

Introduction

Fishery production is the largest sector that creates a greater impact on the economic growth of a country. India is the second-largest fish-producing country and also the second-largest aquaculture nation next to China in the world. Till now in India, the examination of fishes is performed in a traditional way that involves human specialists. A large amount of time and money is wasted on checking the freshness of the fishes by humans. In this paper, we are analyzinga way to detect thequalityof sardine fish based on its size, shape, and color. The testing of fishes should be done carefully because these are very fragile items.

The traditional method of inspecting the fish quality is done by sensory evaluation of the gills and eyes of the fish. Though these factors can be observed through the naked eye,it is not an easy task for a consumer to detect the quality of the fish as it requires highly skilled personnel.These quality measures are mapped into an automated system that could make the testing process faster with no errors. Recently, deep learning techniques have been found progressively useful in the food processing industry.

Figure 1.Images of fresh and stale sardine fish

(2)

Literature Review

[1]

In this paper, an android application was developedthat automatically identifies the fish freshness by using image processing; the app categorizes the fish ranging from level 1 (stale) to level 5 (fresh) based on its freshness by using the RGB colorvalues of the gills and eyes. It also determines whether the fish is in the consumable state or not. 800 images of gills and eyes of nearly 30 fish samples per species are given asinput to the proposed system. The proposed system uses a feed-forward neural network that learns the features iteratively from the given images. Also, it helps to identify the categories of fish and their freshness.

[2]In this paper, the proposed system detects shrimp freshness by using the Indian rhododendron (Melastomamalabathricum) L-based label indicator. The amount of flavonoids within the extract allows changing the labelcolor from red to grey due to the interaction of the label with the OH- groups which comes out from the shrimp spoilage process. The changes in the label colour indicate the shrimp freshness.

Toimprovise the effectiveness of detecting the quality of fishes, the classification task is conducted by using the k-nearest-neighbors algorithm, which is provided with an image processing mechanism that involves sorting of color quantization. The accuracy obtained is 71.9%. Hence the label indicators are very promising to be developed.

[3] This paper determines the correlation that exists between the statistical wavelet coefficients and freshness in the stored fish. Features Extraction plays a vital part in detecting the quality of fish.The extracted features are experimented with the conventional chemical laboratory-based method.This image processing-based algorithm serves as a robust diagnostic tool for food quality inspection and control. It creates a new dimension of research in testing the quality of food items in real-time applications.It acts as a monitoring tool that helps to ensure the qualityof food provided to the consumer. A future perspective also concerns the toxicity in the fish samples.

This paper also suggests a prototype may be designed and put into practice forprocessing different types of fishes.As the numbers of days pass on,the values of wavelet coefficients are increased which results from the decrease in the quality of the food item. The proposed work reveals that the quality of fishis inversely proportional to the selected wavelet coefficient. The Region of Interest (ROI) of a fish in an image is extracted by thesegmentation process then grouping the fishes to their freshness levels byusing k-Means Clustering. The proposed workgives accuracy 92.4%.

[4] In this paper, the system uses three feed-forward neural networks to classify the fish samples to their freshness levels. The effectiveness of the system is evaluated with the representatives from the Bureau of Fisheries and Aquatic Resources. On observation, each fish species differs in their corresponding RGB colour values of its classes. Constant illumination is an important factor in image processing. Hence the Light source is maintained as an external means whenever the images are captured. The whole process is implemented in MATLAB and the Graphical User Interface is created in Android Studio. The techniques used are K-means clustering and Canny edge detection and obtained 100% accuracy in classification

[5] In this paper, the classification of milkfish freshness is based upon the results of correct fish-eye segmentation. But the problem in the image segmentation isthat multiple objects have similar characteristics and similar greyscale intensity. Image Segmentation is performed by using K-Means clustering provides multiple objects on segmentation. Above all, the eye feature is the most important feature to be considered. The classical K-Nearest Neighbour (KNN) is sensitive to noise when K has a low value. When K has a high value, then there is more chance for the objects that fall to the primaryclass. The proposed work uses Cosine KNN (CosKNN) to resolvethe problem present in the classical KNN. Instead of the classification results based on the majority votes of the nearest neighbor, CosKNNprovides a soft value that represents the belonging level of every class to the testing data. The experimental result proves that the CosKNNgives superior performance compared to classical KNN in terms of precision and recall.

[6] In this paper, the twosignificant factors that have a greater impact on finding the quality of fish are the retention time and medium used for the storage procedure. The standard of the fish sample could also be decreased every other day until it finally reachesthe consumers. This paper suggests the automated method for classifying tilapia fish freshness. The proposed method is implemented using real-time dataset images captured

(3)

on several different days and aiming to find the quality of fish taken at an appropriate time interval.

[7] This paper discusses the merits and demerits of the current fish freshness detection techniques. Various methods such as the Sensory evaluation method, microbiological method, physical and chemical method are used to find the quality of fish and yielded accuracy of 86.3%.

[8] This paper demonstrates the design methodology for An Intelligent Fish freshness system for real-time classification of the freshness of fish using an Artificial Neural Network (ANN). Automated Fish freshness analysis and detection is the critical task in the fishery industry. This research is a preliminary work to store and analyze the sensory data using ANN. A specific network is trained for the quality assessment of fish freshness.

The proposed system provides rapid response because a trained neural network arranges data in real-time. It has been successful in identifying the number of days after catching the fish with an accuracy of up to 99%.

Methodology

The images of the sardine fish are collected. Then the fish images are scaled to the standard size 224*224.

Further, all the RGB images are converted to Grey Scale images. After color transformation, normalize the data from 0-255 into the range [0, 1]. Then partition the dataset into the training set, test set, and validation set using Keras. During training, the network learns the characteristics of images.The validation data assess t classifier model how well it performs given the training dataset. During testing, unseen images are given as input to the classifier to categorize the images.The proposed system uses Deep Convolution Neural Network (CNN) through which detailed features are extracted by superimposing the filter over an image during convolution operation[15][16]. Irrelevant and noisy features are removed during the pooling operation. After the pooling operation, the representative features are fed as an input to a fully connected layer that uses the soft-max function to compute the maximum likelihood of a fish sample belongs to a particular category.The proposed system is used for finding the quality of fish whether it is the stale or fresh fish image.

Fig.1.Block Diagram of the proposed system

Convolution Neural Network

CNN is a specialized type of ANNs particularly used for image analysis. The computer sees the image as a matrix of numbers where each numberrepresents a single pixel. The relationship between the pixels (values) must be preserved even after the image is given as an input to the network. A convolution neural network retains the spatial information between the pixels. Different mathematical operations are piled on top of each other layers to transform the input into the output. A detailed description i.e. about the architecture of convolution neural network is given below.

The Architecture of Deep CNN Model

Our proposed system consists of two-layered deep neural network model architecture. The first layer comprises Pre-processing :

Image Re-Scaling Color

Transformation

Data Partitioning

Training Data

Test Data

DCNN Classification

Fresh Fish

Stale Fish

(4)

a convolution layer with maximum pooling to increase the structural ability, whereas the second layer comprises fully connected layers. The neural network has four layers. The input consists of 160×160×4 neurons, which represents the RGB(Red, Green, and Blue) value of an image. The first convolution-pooling layer applies a kernel of size 4×4 with a stride length of 1 pixel to mine 32 feature maps, followed by a maximizing the pooling operation managed in a 2×2 region, the second and third convolution-pooling layers use the same 4×4 kernel resulting in 64 and 128 feature maps respectively whereas other parameters remain unchanged. The fourth layer has a fully connected layer that acts as an output layer that classifies 16 different categories of sardine fishes.

The three convolution-pooling layers use ReLUas their activation function.

There are threeoperations in the Convolution Neural Network; they are convolution, pooling, and classification.Convolution operation in CNN is used to mine features from the input image. It alsopreserves the spatial informationbetween pixels and learnsthe characteristics of an input image by extracting its features. The model uses a 4×4 filter that slides over the image to obtain convolved properties. [9][13]

The two-dimensional convolution operation is given by, (X*K)(i, j)=ƩmƩnK (m, n) X (i-m, j-n)

Where X represents the 2d matrix of an original image, K represents a 3*3 matrix of a filter and * represents discrete convolution operation performed when filter slides over the spatial region of an original image.

Rectified Linear Unit (ReLU) is used as an activation function to perform a non-linear operation activation function for triggering the network. The main objective of ReLU is to perform a non-linearity function in the Convolution Neural Network.ReLU activation function trains the neural network many times faster without compromising generality, precision, and accuracy. The advantage of ReLUover sigmoid and tanh function is its unsaturation of the slope, which highly accelerates the concurrence of stochastic gradient descent. Another aspect related to tanh and sigmoid function is that it demands costly operations.

f(x) = x for x>0

= 0 for x<0 [9]

Fig.2 CNN Architecture

The fourth layer; the fully connected layer uses soft-max as their activation function to classify the fish samples to their freshness levels.

It is defined as follows,

[10]

Where zi is the input vector and denominator is the normalizing factor which is equal to 1

(5)

Dataset Collection

Dataset collection is a very important activity for the research as it is much expensive and harder to generate.The snapshots of the sardine fish are captured at a resolution of 320×258×3 pixels by using High definition camera. From an experimental perspective, fish sample imagesare reduced to 224×224×3 resolution.

The dataset consists of 1049 images of fresh sardine fish and 1078 images of stale sardine fish. All images are stored at 16 bits per channel in RGB color space. The Images have huge differences in terms of quality and brightness. Illumination is alsopart of these variations in the images. Images are also taken with different conditions i.e. with lights of the room on and off, changing the position of the camera at different angles, open windows, and closed windows scenario. For a real-time application, it is very essential to manage brightness variation, camera snapshot artifacts, shadows, and specular reflections. The datasets are constructed by considering all these real-world challenges.

Fig.3. Images of Fresh Sardine Fish

Fig.4.Images of Stale Sardine Fish

Evaluation Measures

The proposedsystemis assessed by measuring the Accuracy, Sensitivity, Specificity, Positive Prediction Value, Negative Prediction Value, and f1-score.

Accuracy is the ratio between the number of images that are correctly classified and the total number of images.

It is given by,

Accuracy= (TP+TN)/ (TP+TN+FP+FN) [11]

TP is correctly identified as Stale fish TN is correctly identified as Fresh fish

(6)

FP is incorrectly identified as Fresh fish FN is incorrectly identified as Stale fish

Sensitivity or Recall is the ratio between the correctly identified stale labeled fish images by our proposed systemandthe total number of stale fish images in reality. It is given by,

Sensitivity =TP/ (TP+FN)

Specificity is the ratio between thecorrectly identified fresh fish labeled images by our proposed systemand the total number of images that are fresh fish images in reality.

Specificity =TN/ (TN+FP)

Positive Prediction Value (PPV) is the ratio between the correctly identifiedstale labeled fish imagesby our proposed systemand all stale fish images in reality. It is given by,

PPV=TP/ (TP +FP)

Negative Prediction Value (NPV) is the ratio between the correctly identified stale fish labeled images by our proposed system to all the fresh fish images in reality. It is given by

NPV= TN/ (TN +FN)

F1-score is the harmonic mean of precision and recall. It is given by,

F1-score= (2*Precision*Recall) / (Precision + Recall) [12][14]

Result

The binary classification of the proposed model predicts the class of an image where 0 represents stale fish and 1 represents fresh fish. The performance of our proposed model is given in the table1. Also, the performance measures such as accuracy, sensitivity, specificity, Positive Prediction Value(PPV), Negative Prediction Value(NPV), and f1-score are evaluated for the proposed model. It gives a good result in terms of performance measures and it is depicted in the table.1.The corresponding performance measures are plotted in fig.5

Table.1 Performance of the proposed DCNN classifier model

Performance measures DCNN model

Sensitivity 96.2%

Specificity 92.3%

Positive Predictive Value 92.6%

Negative Predictive Value 96%

Accuracy 99.5%

F1-score 94%

Fig. 5. Performance measure of the proposed DCNN classifier model 85.00%

90.00%

95.00%

100.00%

105.00%

Deep Convolution Neural Network model(DCNN)

DCNN model

(7)

Conclusion

The Food Production industry plays a major part in the economical growth of a country. Our country is the second-largest producer of fishes.Particularly in the fishery industry, it is a requirement to detect the quality of fish by finding its freshness for commercial purposes.So far the testing process involves human specialists.Hence it triggers us to automate the testing process.There are limited existing works and mostly the existing works use machine learning algorithms to detect fish freshness.We find deep convolution neural network provides remarkable results in image processing.Hence our proposed system uses a deep convolution neural network to detect the freshness of the sardine fish samples and classify fish samples as fresh fish and stale fish. DCNN; an automated detection system is implemented, evaluated, and obtained remarkable results in the performance measures such as accuracy 99.5%, sensitivity 96.2%, specificity 92.3%, PPV 92.6%, NPV 96%, and f1-score 94%.This work could be extended by developing a system that determines the grading levels of the fish freshness overdifferent days. It requires more samples, more features range from low level to high level are to be extracted and images are tobe captured at different angles with constant illumination. Also, we plan to implement different classification algorithms for detecting sardine fish freshness.

REFERENCES

[1] Ni Made SatvikaIswari, Wella, Ronny, “Fish Freshness Classification Method Based On Fish Image using k-Nearest Neighbour", 4th International Conference on New Media Studies, 2017.

[2] AlefiaNoor,Evi J, AisyahD.A.T.Safitru, Mustari, YuantTiandho, "Melastomamalabathricum Extracts - based indicator for monitoring shrimp freshness integrated with classification technology using nearest neighbor algorithm".

[3] Monika Arora, ParthasarathiMangipudi, M.K.Dutta, RadimBurget, "Image processing based automatic identification of freshness in fish Gill tissues", International Conference on Advances in Computing, Communication Control and Networking (ICACCC2018)

[4] Ian C.Navotas, Charisse Nadine V. Santos ,Earl John M.Balderrama, Francia Emmanuelle B.Candido,Aloysius John E.Villacanas, Jessica S.Velasco,"Fish identification and freshness classification through image processing using artificial neural network ",Asian Research Publishing Network (ARPN)2016-2018.

[5] Ekoprasetyo,R. Dimas adityo,Ranipurbaningtyas," Classification of segmented milkfish eyes using cosine k-nearest neighbor”, International Conference on Applied Information Technology and

[6] Jun Gu, Nan He, XiaoxueWu,"ANew Detection Method for Fish Freshness ", Seventh International Symposium on Computational Intelligence and Design,2014.

[7] DharmendraKumar,Shivesh Kumar, Dr.N.S.Rajput, "An Intelligent System For Fish Freshness Quality Assessment Using Artificial Neural Network”, International Journal of Creative Research Thoughts.

[8] Fatima M. Salman1, Samy S. Abu-Naser1, Eman Alajrami2, Bassem S. Abu-Nasser1, Belal A. M.

Ashqar1 "COVID-19 Detection using Artificial Intelligence" Vol 4, Issue:3 March 2020

[9] Ioannis D. Apostolopoulos1 · Tzani A. Mpesiana2,”Covid‑19: automatic detection from X-ray images utilizing transfer learning with convolution neural networks”https://doi.org/10.1007/s13246-020- 00865-4

[10] Ezz El-Din Hemdan1, Marwa A. Shouman1, and Mohamed Esmail Karar 2,"COVIDX-Net: A Framework of Deep Learning Classifiers to Diagnose COVID-19 in X-Ray Images”

(8)

[11] Tulin Ozturka, Muhammed Talob, Eylul Azra Yildirimc, Ulas Baran Baloglud, Ozal Yildirime,*, U.

RajendraAcharyaf, G,H,”Automated detection of COVID-19 cases using deep neural networks with X- ray images”https://doi.org/10.1016/j.compbiomed.2020.103792, April 2020

[12] TalhaBurak Alakus, Ibrahim Turkoglu, "Comparison of deep learning approaches to predict COVID-19 infection July 2020

[13] Prabira Kumar Sethy a, SantiKumariBehera b," Detection of coronavirus Disease (COVID-19) based on Deep Features”

[14] MdZabirul Islam, MdMilon Islam, Amanullah Asraf, "A combined deep CNN-LSTM network for the detection of novel coronavirus (COVID-19) using X-Ray images", August 2020

[15] Rajeswari C., Sathiyabhama B.,Devendiran S., Manivannan K. Bearing fault diagnosis using wavelet packet transform, hybrid PSO and support vector machine,Procedia Engineering, ,Vol.97 (1) PP:1772- 1783,2014 (CSE).

[16] Rajeswari C., Sathiyabhama B., Devendiran S., Manivannan K. A Gear fault identification using wavelet transform, rough set based GA, ANN and C4.5 algorithm. Procedia Engineering, Vol – 2 PP:

338-344, DOI : 10.1016/j.procs.2010.11.044,2014

[17] Asraf Yasmin, B., Latha, R., & Manikandan, R. (2019). Implementation of Affective Knowledge for any Geo Location Based on Emotional Intelligence using GPS. International Journal of Innovative

Technology and Exploring Engineering, 8(11S), 764–769.

https://doi.org/10.35940/ijitee.k1134.09811s19

[18] A.M. Barani, R.Latha, R.Manikandan, "Implementation of Artificial Fish Swarm Optimization for Cardiovascular Heart Disease" International Journal of Recent Technology and Engineering (IJRTE), Vol. 08, No. 4S5, 134-136, 2019.