Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Divide, Conquer, and Distill: Unified Sample Selection and LLM Distillation for Noisy Long-Tailed Learning

AbstractThere is a rising interest in further exploring learning with noisy labels on textual data. Existing solutions often assume the ground truth classes of the training data are balanced, overlooking the long-tailed distribution prevalently encountered in real-world data, which can significantly harm the effect of denoising strategies. Meanwhile, misidentifying clean samples further compromises the model’s learning of long-tailed data. To tackle this issue, we propose a novel knowledge distillation framework for learning with noisy labels under long-tailed distribution (LNL-LT). Specifically, we first introduce a divide-and-conquer sample selection strategy to perform head-tail class division and better separate clean samples from noisy samples within these classes, especially for the tail classes. Then, we employ LLM knowledge distillation to generate pseudo-labels and new samples for tail classes, enhancing pseudo-label quality while mitigating the effects of sample scarcity caused by the long-tailed distribution. Experimental results on both synthetic and real-world datasets demonstrate that our method surpasses current state-of-the-art methods.

More from EMNLP 2026

All EMNLP 2026 papers →