Facial Emotions and Behaviour Monitoring System using DNN

Abstract

In this paper, Machine Learning Algorithms are used to implement the proposed approach to identify social distance, face masks, drowsiness detection, age-gender detection, and emotion detection. While dealing with social distancing initially, we need to detect humans’ faces, which are available by using COCO (Common Objects in Context) datasets, and later on, polygon-shaped ROI (Rectangular-region of Interest) is warped with a rectangle, which helps to find the distance from each centroid (person). Similarly, we predict the facemask, age-gender, emotion, and drowsiness altogether using frontal-face detection and eye-detection via haarcascade dataset loaded into Convolutional Neural Network (CNN) to train and test the models on colour mapped images. In the proposed model, we are using machine-learning techniques such as linear discriminant Analysis (LDA), Independent Component Analysis (ICA), Principal Component Analysis (PCA) for age-gender detection and emotion detection, K-Nearest Neighbours (KNN) for Social Distancing, and Support Vector Machine (SVM) for facemask detection and drowsiness detection. The accuracy of proposed system depends on frame (i.e., 88.2%, 89.7%, 95.1% and 98.3% in 0~0.2s, 0.2~0.6s, 0.6~1s, >1s time windows respectively). The accuracy even depends upon the distance away from the camera (i.e., 60.4%, 73.9%, 89.3%, 95.2%, and 62.2% in >15, 15~10, 10~6, 6~0.5, <0.5 meters respectively). The resultant average accuracy of all the models is 96.3%, which is capable to predict various tasks as said above. This complete model is made accessible to users via a standalone software/Desktop GUI. The proposed approach is promising for performing all the tasks and activities more accurately and efficiently.the systemic health of the patient and avoiding possible drug interactions