• Nu S-Au Găsit Rezultate

View of Monitoring of Emotional Status and Identity Verification of a Vehicle Driver

N/A
N/A
Protected

Academic year: 2022

Share "View of Monitoring of Emotional Status and Identity Verification of a Vehicle Driver"

Copied!
9
0
0

Text complet

(1)

Received 20 January 2021; Accepted 08 February 2021.

http://annalsofrscb.ro 884

Monitoring of Emotional Status and Identity Verification of a Vehicle Driver

Dr.S.S Sivaraju1, Dr.M.Meenakumari 2, M.Thiruveni 3, S.Suryaprakash4, Dr.P.Vinoth Kumar5

1 Professor & Head, Department of Electrical and Electronics Engineering, RVS College of Engineering and Technology, Kannampalayam, Sulur, Coimbatore-641402.

2 Professor, Department of Electronics and Communication Engineering, Sri Ranganathar Institute of Engineering and Technology, Coimbatore.

3Associate Professor, Department of Electronics and Communication Engineering, PSNA College of Engineering and Technology, Dindugul.

4 Assistant Professor, Department of Electrical and Electronics Engineering, Kumaraguru College of Technology, Coimbatore.

5Associate Professor, Department of Electrical and Electronics Engineering, Sri Krishna College of Engineering and Technology, Coimbatore.

ABSTRACT

Transportation frameworks are a fundamental piece of human exercises. We as a whole can be a survivor of enthusiastic changes while driving, just following a too brief night rest, adjusted state of being or during long excursions. The vibe of rest decreases the driver's degree of cautiousness creating hazardous circumstances and expands the likelihood of an event of mishaps. Driver laziness and exhaustion are among the significant reasons for street mishaps. Consistently, there is an expansion the quantity of passing’s and fatalities wounds universally. In this specific situation, it is critical to utilize new advances to plan and construct frameworks that can screen drivers and to quantify their degree of consideration during the whole interaction of driving. Here we consider vehicle safety as well as vehicle driver identification. Among the individual distinguishing proof strategies, face acknowledgment is perhaps the most tended to methods because of its impartiality as a biometric characteristic. The extraordinary exploration endeavours on face acknowledgment have given huge enhancements in face acknowledgment. Here we use a smart band along with a dash cam to monitor the driver.

A vehicle can identify and verify the driver by matching the facial biometric trait stored in the smart watch and the once collected from the detected face. The matching of both verifies the smart watch owner and the vehicle driver is the same. The readiness of the driver is controlled by observing the anomalies in the pulse change and gives cautioning alarms. Thereby working as both a security and safety feature.

Index Terms:

Driver monitoring, Face detection and recognition, Fuzzy Logic, Haar cascade classifier

1.Introduction

Driving securely is a vital issue that is as a rule truly considered for research and mechanical applications. Driving is a summed-up movement that at times infers a danger for the driver and outsiders. It is affected by the climate, the street foundation, the mechanical-electrical status of the vehicle, and the well-being and passionate condition of the driver. To manage some regular reasons for mishaps, the auto industry is creating inventive approaches to give security through frameworks called advanced driver assistance systems which are valuable for diminishing auto collisions and advancing more smooth and productive transportation [1].In general, help frameworks screen outer components like the street, the measure of light, the distance to different vehicles, and so forth These sorts of frameworks give a few alarms to the driver when there is a danger. A few organizations in the car area have been executing dynamic control frameworks for driver help, and some very good quality vehicles that have consolidated dynamic control frameworks in the suspension, or in the directing: counterfeit vision frameworks for line following, or crash shirking frameworks. Also, safe driving necessitates that the driver's enthusiastic state supplements abilities, for example, the correct judgment of the traffic, readiness, suitable dynamic and correspondence with different drivers [2].

(2)

Received 20 January 2021; Accepted 08 February 2021.

http://annalsofrscb.ro 885

This paper proposes a structure for driver wellness acknowledgment utilizing a wrist band, for identifying the heartbeat of the driver and face discovery. The sharpness of the driver is dictated by observing the variations from the norm in the heartbeat rate change and gives cautioning alarms. In the event that the driver seems, by all accounts, to be encountering physical illness, our framework can caution the driver to his condition and alarm the driver in charge. Thereby working as both a security and safety feature.

Apart from driver physical and mental conditions, another key reason for accidents is an unauthenticated driver. The authorized person will take a nap and may ask the unauthorized person to drive the truck or he will outsource it to unauthorized drivers who are inefficient in driving it. This can be overcome by using the proposed technology. Face detection algorithm can be programmed to control Engine Control Unit (Car or Truck). By using this technology the system will be monitoring the person at several instances during the journey [8].

In this proposed system, whenever a person tries to start the engine, it will authenticate the person with the preloaded secured data that is stored in the system. The system will compare the data from the wristband and the acquired data from the onboard camera.

This biometric technology can be also used to aid keeping away from auto collisions as well as in forestalling vehicle burglaries also. Customized settings in a vehicle is a convenient component once the framework distinguishes the driver, it can consequently change the mirrors, the seat and the controlling wheel, and turn on the most loved radio broadcast. Taken one level higher, the framework could likewise be utilized to change further developed execution settings, for example, choke reaction, gearshift examples and suspension solidness [10].

Face detection and identification is a difficult zone in computer vision and example distinguishing proof due to its shifted outward appearances, stances and enlightenment. Face acknowledgment is extremely fundamental in territories like access the board, human-computer communication, creation control, e-learning, exhaustion driving acknowledgment and enthusiastic robot. We are using Haar-cascades to find the Region of Interest (ROI). For the recognition of the face, HAAR highlights are the primary piece of the HAAR Cascade Classifier HAAR highlights are utilized to distinguish the presence of a component in the given picture.

Ada boost a machine-learning algorithm which helps to find the best features are used along with HAAR-cascade algorithm to detect the human face [11].

Facial features are required to recognise human faces, for this, facial features must be extracted from the Region of Interest. Facial element extraction is the way toward separating face part includes like eyes, nose, mouth, and so on from the human face picture. Facial element extraction is a lot of significant for the instatement of handling strategies like face following, outward appearance acknowledgment or face acknowledgment. Among every single facial element, eye limitation and identification is fundamental, from which areas of any remaining facial highlights are distinguished. Geometric Shaped Facial Feature Extraction for Face Recognition (GSF2EFR) is utilized for recognizing the specific individual by finding the middle and corners of the eye utilizing eye discovery and eye confinement modules [13, 16, 17, 18]

2. Literature survey

In the previous few decades, various methods have been proposed to screen the driver's feelings.

A portion of the strategies for driver feeling acknowledgment, for example, [1] and [2] depend on observing physiological signals, for example, EEG, ECG, electro-dermal movement,

(3)

Received 20 January 2021; Accepted 08 February 2021.

http://annalsofrscb.ro 886

electromyography, and so on.Camera-based Geometric Shaped Facial Feature Extraction For Face Recognition (GSF2EFR) is utilized for recognizing the specific individual by finding the middle and corners of the eye utilizing eye discovery and eye confinement modules feeling acknowledgment frameworks are, in this way, more qualified for checking the driver's feelings, since it is non-meddlesome and doesn't need the dynamic interest of the driver. These techniques depend on breaking down the facial pictures of the driver and performing face appearance acknowledgment to screen the driver's feelings.

Spiros et al. [3] proposed a structure for feeling acknowledgment utilizing Facial Animation Parameters (FAP) and a neuro-fuzzy rule-based framework to classify the feeling. Wang et al. [4]

track a face in each edge and discover the highlights of eye, mouth and head. At that point these facial highlights are examined to recognize the driver feelings.

Gao et al. [5] proposed a constant structure to recognize the enthusiastic condition of a driver by dissecting outward appearances, wherein, a face tracker is utilized to follow facial milestones, SIFT descriptor is removed for every milestone and straight SVM is utilized to to classify the expression.

Appearance characterization is an exceptionally testing issue additionally on the grounds that a slight variety in articulations may bring about an entirely unexpected feeling due to the similitude of outward appearances when changed feelings are being felt. For instance, outward appearances on account of trouble or dread are very much like Hong-Wei et al. [14] use transfer learning in CNN with supervised fine-tuning to characterize the outward appearance.

They follow two phases of calibrating, at one phase they tweak the organization on one dataset and at the subsequent stage, and they train the organization on preparing information of other datasets. Mollahosseini et al. [15] utilized two convolution layers followed by max-pooling and afterward four initiation layer to arrange the outward appearance. Lopes et al. proposed a five- layered CNN with two convolution layers, two sub-examining layers and one completely associated layer and apply pre-preparing for highlights explicit for articulations to conquer the issue of absence of accessibility of huge datasets for outward appearance. Heechul et al. utilize deep networks for facial expression recognition. In the principal profound organization first deep network, they separate worldly appearance highlights from picture arrangements while in other profound organizations, fleeting calculation highlights are extricated from face fiducially focuses.

At last, these two models are joined for characterizing the outward appearances.

A. Driver Monitoring System

Driver observing is partitioned into conduct checking, physiological checking, and feelings checking.Driver conduct is a dubious boundary portrayed by consistent association between the driver, the vehicle and the climate [10]. Probably the most widely recognized driver unwanted states are flushed, delinquent, rash, exhausted/lazy and satisfactory driving. For example, analysts have found that a forceful driver has more unsurprising conduct than a non-forceful, and this subsequent gathering is more responsive to the activity of the framework [7]. Checking different drivers is additionally a decent alternative to give admonitions to the subject and keep away from mishaps. A few creators propose the utilization of remote sensors to gather data about the driver, vehicle state, and encompassing changes to find driver conduct and afterward send information to different drivers, shrewd vehicles, or even to the foundation. The manner by which a vehicle is driven relies straightforwardly upon activities taken by the driver. Kondyli proposed a decent investigation of driver's developments inside the vehicle by a 7-point human skeletal model.

Some new techniques have been suggested that utilization numerical models like Markov anchors

(4)

Received 20 January 2021; Accepted 08 February 2021.

http://annalsofrscb.ro 887

or Bayesian insights to foresee a driver's proposed activities across traffic circumstances. With these forecasts, more successful assistance can be given to drivers and the vehicle's overall degree of knowledge is higher. As of late, dynamic has been concentrated by Tang to comprehend the conduct of drivers when traffic lights change at rapid crossing points. One outcome was a prescient model to comprehend the progression of traffic [9]. This is the most considered checking framework because of the quantity of mishaps got from sleepiness, weariness, and interruption while driving.

3. Proposed system

The proposed approach consists of face detection and recognition, by the combination of the deep networks, and driver capability detection by measuring the heartbeat of the driver. By the combination of face detection and driver capability observation the safety and security of the vehicle in which this proposed technology incorporated could be increased.

The proposed system has two portions. One is a wearable band and other is an on-board system.

The wearable band suggested here looks like a fitness band or a smart watch. The wearable band will be capable of measuring heart rate like in a fitness band and acts as a digital license for the driver. That is, all the information in an issued vehicle driver’s license will be stored in the wearable device along with face feature information of the driver. Which will be later used for driver identification.

The prototype implementation of a wearable device includes a node MCU and a heart rate measuring sensor. The on-board unit is capable of driver identification and an intelligent alert system

Fig.1. Driver monitoring system

The driver observing camera screens the readiness and state of the driver. It can distinguish interruption, sluggishness, and micro sleeps and can alarm the driver as expected. Facial acknowledgment makes it feasible for the framework to recognize the driver with supreme conviction. In view of the put away driver profile, the framework can then naturally set individual solace and comfort settings, for example, ideal seat and mirror positions, the most loved radio broadcast, or the favored inside temperature.

The camera is situated so that the driver seat is in its field of vision. The framework can recognize the presence of the driver and would then be able to give more explicit updates. Driver identification is possible by the combined effort of the wearable device and the on-board unit.

The wearable device provides the face feature information of the driver to the on-board unit through wireless means. The wireless means used here is Wi-Fi. The on-board unit collected the

(5)

Received 20 January 2021; Accepted 08 February 2021.

http://annalsofrscb.ro 888

data received and matches it with the face captured by the camera. If the match is true, then the driver is identified and verified.

The control of the caution and start frameworks is finished by the u rationale. Fuzzy rules are characterized for different conditions and the framework working by reference to the set fuzzy guidelines.

A. Face Detection and identification

Face detection is a piece of face recognizable proof. At the point when we see at the individual's face, can get data, for example, the demeanor, gender orientation, age and nationality. Face recognition is helpful in numerous applications, for example, reconnaissance framework, human- machine cooperation, biometrics, gender order and so forth For individuals face recognition is a simple assignment however face location is a serious extreme errand for a PC. An advanced picture is comprised of a limited number of components every one of which has a specific area and worth. These components are known as pixel and picture component. These components take cooperation to discover the face. The face recognition technique can be extensively characterized into two classifications: Appearance based methodology and highlight based methodology. In the appearance based methodology, the entire picture is utilized as a contribution to the face indicator.

In element based methodology, face discovery depends on the highlights separated from a picture. Highlights can be for example skin tone or edges and now and again they have an information on the face math. The appearance based methodology which we utilized in this paper can possibly recognize the face from a picture utilizing haar course classifier.

As far as speed and unwavering quality for face identification from a picture, haar course classifier is probably the best identifier.

B. Face Detection

Open Source Computer Vision Library (Open CV)[10][12] is utilized to actualize the haar course classifier. For the discovery of the face, haar highlights are the principle part of the haar course classifier. Haar highlights are utilized to identify the presence of a component in a given picture.

Each component brings about a solitary worth which is determined by deducting the amount of pixels under a white square shape from the amount of pixels under a dark square shape Haar-like features are the rectangle shape highlights for quick face identification. Some haar like element is appeared in Fig.2.

Fig.2.Haar Features

The Haar highlight begins filtering the picture for the recognition of the face from the upper left corner and closures the face identification measure base right corner of the picture. The picture is examined a few times through the Haar like highlights to identify the face from a picture [10].

In that cascade, a picture will be a human face in the event that it passes all the stages. On the off chance that it isn't passed any of the stages it implies the picture is definitely not a human face.

Line Feature Four-rectangle feature

Edge Feature

(6)

Received 20 January 2021; Accepted 08 February 2021.

http://annalsofrscb.ro 889

Then the detected human face is cropped from the image acquired and this cropped image is used for the feature extraction process, which is the next step in face recognition.

C. Feature Extraction and Identification

Facial element extraction is the way toward removing face segment highlights like eyes, nose, mouth, and so forth from the human face picture.

Here we used the Geometric Shaped Facial Feature Extraction for Face Recognition system (GSF2EFR)[13] for distinguishing proof and confirmation reason. The information picture must be recognized from the given data set. In the face recognition measure, the face picture is considered as info and dependent on its yield the face gets distinguished for recognizing the specific individual Face Features from the IoT.

The framework proceeds in a top-down style. From the distinguished eye places, it acquires eye, nose and mouth sub-pictures with mathematical contemplations and concentrates the fiducially focuses. At last, from Geometric distances between fiducially focuses, from the face of the person are compared with the fiducially features that are stored in the IoT. If the fiducially features are from the IoT and acquired fiducially features from the onboard camera are similar, then the diver gets identified. As the identified driver is authorized to drive the vehicle, the driver can start and drive the vehicle. In the event that the fiducially highlights are not coordinated, at that point the driver is set apart as unapproved by the framework and the driver is limited from driving the vehicle.

D. Simulation Results

Simulation of the Fuzzy Logic Controller has been carried out. Simulation is carried out in Simulink. In the simulation, various input is provided to the system and the systems are supposed to take necessary action depending on the input received. Fuzzy logic includes 0 and 1 as extreme cases of truth but also includes the various states of truth in between. Fuzzy logic seems closer to the way our brains work. We aggregate data and form a number of partial truths which we aggregate further into higher truths which in turn, when certain thresholds are exceeded, cause certain further results such as motor reaction.

The inputs chosen here are whether a face is identified or not, whether the license has expired, then heart rate and the average heart rate for 15min interval and the vehicle speed. The average heart is calculated to get a better idea of the long term physical and emotional status. In here Fuzzy Logic Designer is used to designing the fuzzy interface structure. A double output fuzzy logic controller is used, One output is used to control the ignition relay and the other is used to provide appropriate voice alerts. For the speech, the voice is synthesised from the text, using the Microsoft Win32 Speech API (SAPI).

Rules created for this case are represented in a table format and shown in Table I. The yield of each standard is a fuzzy set gotten from the yield enrolment work and the ramifications technique for the FIS. These yield fuzzy sets are consolidated into a solitary fuzzy set utilizing the accumulation strategy for the FIS. At that point, to figure a last fresh yield esteem, the joined yield fuzzy set is de-fuzzified.

INPUTS OUTPUTS

Face Identified

License Expiry

Heart Rate

Heart Rate Average(

15min)

Vehicle Speed

Ignition

Relay Voice Alert

FALSE x x x x FALSE No Alert

(7)

Received 20 January 2021; Accepted 08 February 2021.

http://annalsofrscb.ro 890

TRUE TRUE x x x FALSE License

Expired

TRUE FALSE Normal Normal Normal TRUE No Alert

TRUE FALSE Low Normal Normal TRUE Pull-Over

Alert

TRUE FALSE High Normal Normal TRUE No Alert

TRUE FALSE Normal Normal High TRUE No Alert

TRUE FALSE Low Normal High TRUE Pull-Over

Alert

TRUE FALSE High Normal High TRUE Slow-down

Alert

TRUE FALSE Normal x Low TRUE No Alert

TRUE FALSE Low x Low TRUE No Alert

TRUE FALSE High x Low TRUE No Alert

TRUE FALSE Normal High High TRUE No Alert

TRUE FALSE Low High High TRUE Pull-Over

Alert

TRUE FALSE High High High TRUE Slow-down

Alert

TRUE FALSE Normal Low High TRUE No Alert

TRUE FALSE Low Low High FALSE Emergency

Stop

TRUE FALSE High Low High FALSE Emergency

Stop

Table I. Rules Table 4. Conclusion

The simulation in Simulink was carried out successfully. The fuzzy inputs were chosen in such a way to reduce complexity, but to produce an optimal result as well. Fuzzy rules were made to cover most of the required voice alert cases and ignition control cases. Currently, a single, double-output multiple-input fuzzy logic controller is used. For further developing the controls, a self-training AI would be better, which will be able to bring about personalised user experiences.

Till now there have been no recognized ways to identified driver till now. This system solves that problem. This will lead to a new feasible way to identify the driver of a vehicle, any point of time. Currently, the law accuses the vehicle registration certificate owner for all the lawbreaking instead of the driver. But this technology will allow targeting the law to the driver who drove the vehicle at that time, instead of the vehicle owner.

References

[1] C.D. Katsis, N. Katertsidis, G. Ganiatsas, and D. I. Fotiadis, toward emotion recognition in car-racing drivers: A biosignal processing approach, IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, Vol. 38, no. 3, pp.

502–512, 2008.

[2] K.H. Kim, S. W. Bang, and S.R. Kim, Emotion recognition system using short-term

(8)

Received 20 January 2021; Accepted 08 February 2021.

http://annalsofrscb.ro 891

monitoring of physiological signals, Medical and biological engineering and computing, Vol. 42, no. 3, pp. 419–427,2004

[3] S.V. Ioannou, A. T. Raouzaiou, V. A. Tzouvaras, T. P. Mailis, K. C. Karpouzis, and S.

D. Kollias, Emotion recognition through facial expression analysis based on a neuro- fuzzy network, Neural Networks, Vol. 18, no. 4, pp. 423–435, 2005.

[4] J.M. Wang, H. P. Chou, C. F. Hsu, S. W. Chen, and C. S. Fuh, Extracting driver’s facial features during driving, ITSC, 2011 14th International IEEE Conference on, pp.

1972–1977, 2011.

[5] H.Gao, A. Y¨uce, and J.P. Thiran, Detecting emotional stress from facial expressions for driving safety, Image Processing (ICIP), 2014 IEEE International Conference on, pp. 5961–5965, 2014.

[6] Abhishek, G.,Venimadhav, S., Naresh Kumar, R., Surbhi, J., Abdulmalik, A., MdAfsarKamal, R.:Context-awareness based intelligent driver behaviour detection:

integrating wireless sensor networks and vehicle ad hoc networks. Int. Conf. Adv.

Comput. Commun. Informatics 2014, 2155–2162 (2014).

[7] Anup, D., Trivedi, M.M.: Examining the impact of driving style on the predictability and responsiveness of the driver: real-world and simulator analysis. IEEE Intell Veh Symp Proc pp. 232–237 (2010).

[8] YinYinAye, Keigo Watanabe, Shoichi Maeyamaand Isaku Nagai. Generation of Time- varying Target Lines for an Automatic Parking System Using Image-based Processing, Proceedings of the 2015 IEEE Conference on Robotics and Biomimetics Zhuhai, China,2015.

[9] Keshuang, T., Zhu Shengfa, X., Yanqing, W.F.: Modeling drivers ’ dynamic decision- making behaviour during the phase transition period: an analytical approach based on hidden Markov model theory. IEEE Trans. Intell. Transp. 17(1), 206–214 (2016).

[10] K. Goyal, K. Agarwal and R. Kumar, "Face detection and tracking: Using OpenCV,"

2017 International conference of Electronics, Communication and Aerospace Technology (ICECA), Coimbatore, 2017, pp. 474-478.

[11] Ogla, Raheem & Ogla, Alsaheb & Abdul Hussien, Abdulmohssen & Mahmood, Maitham. (2017). Face Detection by Using OpenCV's Viola-Jones Algorithm based on coding eyes. Iraqi Journal of Science. 58. 735-745.

[12] Open-source computer vision library, http://sourceforge.net/projects/opencvlibrary/

[13] S. R. Benedict and J. S. Kumar, "Geometric shaped facial feature extraction for face recognition," 2016 IEEE International Conference on Advances in Computer Applications (ICACA), Coimbatore, 2016, pp. 275-278.

[14] H.W. Ng, V.D. Nguyen, V.Vonikakis, and S.Winkler, Deep learning for emotion recognition on small datasets using transfer learning, Proceedings of the 2015 ACM on international conference on multimodal interaction, pp. 443–449, 2015.

[15] A. Mollahosseini, D. Chan, and M.H. Mahoor, Going deeper in facial expression recognition using deep neural networks, Applications of Computer Vision (WACV), 2016 IEEE Winter Conference on, pp. 1– 10, 2016.

[16] Mathankumar M, Suryaprakash S, Thirumoorthi P, Rajkanna U, “Development of

(9)

Received 20 January 2021; Accepted 08 February 2021.

http://annalsofrscb.ro 892

Smart Car Security System Using Multi Sensors”, International Journal of Pure and Applied Mathematics (IJPAM), Vol.117 (22), pp. 19-23, 2017.

[17] Viswanathan T, Mathankumar M, “Automatic Tollbooth Credit System Using Vehicle Detection and Number Identification”, International Journal of Innovative Technology and Exploring Engineering (IJITEE), Vol.8 (2S2), pp. 313-316, 2018.

[18] Viswanathan T, Mathankumar M, Ramya R, “Gesture identification based on neural network”, International Journal of Engineering and Advanced Technology (IJEAT), Vol. 8, (2S), 2018.

Referințe

DOCUMENTE SIMILARE

The healthcare area is actually among the most crucial filed that must be taken care of[17]. Although there are many brand new scientists to update the existing methods in

Feature extraction mainly refers to the geometric measurements of the leaf image obtained. The three features such as colour, texture and shape-based features are

Toate acestea sunt doar o parte dintre avantajele in care cred partizanii clonarii. Pentru a si le sustine, ei recurg la o serie de argumente. Unul dintre ele are in atentie

(2016) have used in their research study two types of edible and also medicinal fungi species, Ganoderma lucidum and Pleurotus ostreatus, and have chosen as a

The rest of the paper is organized as follows: In order to make the paper as self contained as possible, we provide the definitions of semismooth, semi- analytic and

By contrast to Yeats’ central position at the time, as acknowledged agent of cultural power, Joyce’s resistance was catalyzed by the energy of self-exiling –a third space

Rosato, “A 3D facial expression database for facial behavior research,” in 7th International Conference on Automatic Face and Gesture Recognition (FG2006), IEEE

A novel hybrid feature relevance score metric is proposed for preliminary feature selection in driver inattention detection application. The hybrid score is based on