Human Emotion Recognition Utilizing Transfer Learning

Abstract

Facial Emotion Recognition (FER) is a crucial area of research in the field of Computer Vision (CV) and Deep Learning (DL), aiming to identify and classify emotions such as happiness, sadness, anger, and surprise from facial expressions. This study focuses on developing a model for recognizing facial emotions using DL techniques along with Transfer Learning (TL). For this work, we have used CKPLUS dataset. We have used two pre-trained State-of-the-art (SOTA) Convolutional Neural Network (CNN) models, namely, InceptionV3 and MobileNet, to transfer their learning in the task of feature extraction from our image data. For the classification task, we have used extracted features from these models both individually and fusing them together. For classifying these sets of features, we have used multiple classifiers, namely, Logistic Regression, a custom 1-dimensional (1D) CNN, and hard voting classifier that ensembles the decisions of Linear Support Vector Machine (LSVM), Logistic Regression, perceptron, quadratically penalized SVM, and SVM with quadratically smooth loss. Moreover we have utilized a soft voting classifier that ensembles the decision of LSVM, and SVM with quadratically smooth loss. The fused features extracted from InceptionV3 and MobileNet achieve the highest performance when classified using the voting classifier, with a classification accuracy of 98.48%, a weighted precision of 99.53%, a recall of 98.57%, and an F1-score of 98.48%. This work indicates the ensemble learning technique outperforms individual models across the dataset, identifying and classifying human emotion from an image of facial expression more accurately.