A Novel Ensemble Approach for Facial Emotion Recognition and Sentiment Analysis Using Deep Learning and Attention Mechanisms

Abstract

Facial emotion recognition (FER) and text sentiment analysis are complementary affective-computing tasks, but they are often evaluated independently with inconsistent model comparisons. This study evaluates deep visual backbones and text classifiers under a unified reporting framework. For seven-class FER on a processed AffectNet subset, VGG16, ResNet50, InceptionV3, EfficientNetB0, VGG16 with attention, and a hard-voting ensemble of VGG16, ResNet50, and EfficientNetB0 were compared. For binary sentiment classification on 50,000 IMDB reviews, BERT, LSTM, CNN, and SVM were evaluated. The FER ensemble achieved the highest accuracy (95.2%), precision (94.5%), recall (94.8%), and F1-score (94.7%); VGG16 with attention reached 94.6% accuracy. BERT led the text task with 94.9% accuracy and 94.6% F1-score. The findings support model complementarity and attention-enhanced representation learning while clarifying that the image and text pipelines are parallel rather than cross-modal.