MMLoRA: Multitask Memory Parameter-Efficient Fine-Tuning for Multimodal SER

The differences in emotional expression present significant challenges for the development of effective emotion recognition systems. Although large language models (LLMs) have demonstrated strong performance, their generalization in emotion recognition tasks is often limited by variations in individual emotional expression, which remain inadequately addressed even by parameter-efficient fine-tuning techniques such as LoRA. To address these challenges, we proposes a multitask memory parameter-efficient fine-tuning method (MMLoRA) that enhances multimodal SER by incorporating gender as an auxiliary task. The method leverages shared LoRA experts to facilitate information exchange between tasks and employs a mixture of LoRA experts to process task-specific information. Additionally, a memory mechanism propagates task-specific information across layers. Experimental results demonstrate that the proposed MMLoRA significantly improves emotion recognition performance compared to vanilla LoRA.