Speech Mutil-label Emotion Recognition Using Asymmetric Class Loss Function Based on Effective Samples

Although significant progress has been made in Speech Emotion Recognition (SER), most current research focuses on single-label emotion classification, while multi-label SER faces low classification accuracy due to the long-tail data distribution (e.g., many happiness samples and few fear samples). To address this, we propose an Asymmetric Category Loss Function based on Effective Sample Selection (ER-ASCL) to alleviate the data imbalance. This method adjusts the asymmetric loss by calculating class weights from effective samples and introduces a label-guided matrix to enhance the relationships between emotion categories, improving classification accuracy. Experimental results show that on the CMU-MOSEI multimodal dataset, using the audio modality, the model achieved 53.5% accuracy, a 2.1% improvement over the best existing method. On the Chinese multi-label emotion dataset CNSCED, the model achieved 69.23% accuracy, outperforming existing multi-label classification loss functions.