A Hybrid Multimodal Emotion Recognition-Based Music Recommendation System Using Cnn And Dbn

Abstract

Emotional responses are central to everyday life and strongly shape what people choose to hear in music. Many classic music recommender tools focus on style labels, past listening patterns, or user scores, while ignoring the listener’s moment-to-moment feelings. This work introduces a hybrid, multimodal, emotion-aware music recommendation system that uses Convolution Neural Networks (CNN) together with a Deep Belief Network (DBN) to improve recommendation precision and tailoring. Earlier recommendation methods largely depend on collaborative filtering or content-driven techniques, yet they frequently miss the listener’s immediate effect, producing suggestions that might not fit the user’s present mood. To reduce this gap, the suggested approach combines several input channels---including facial cuts and spoken-audio signals—to estimate the user’s emotion more reliably and accurately. Within the proposed combined design, a CNN is applied to learn rich spatial and temporal representations from visual inputs (facial expressions) and sound inputs (speech), capturing subtle patterns linked to distinct affective conditions. Next, features from each channel are handled and merged, and then a DBN carries out higher-level abstraction and assigns the unified representation to discrete emotion classes (for instance: happy, sad, angry, neutral, calm). The DBN’s stage-by-stage unsupervised pretraining, followed by supervised adjustment, supports learning complicated non-linear links between multimodal feature sets and emotion tags, raising classification performance compared with single-modality methods.