Skip to content
conftrace_
2026 EMNLP EMNLP 2026

End-to-End Argument Mining with LLMs: Representation Insights into Multitask Learning

AbstractArgument mining (AM) traditionally relies on multi-stage pipelines that train separate models for each sub-task, making end-to-end AM challenging and often unstable under multitask learning. In this work, we revisit multitask end-to-end AM in the context of large language models (LLMs) and show that multitask learning behaves fundamentally differently from pre-LLM settings. Across multiple datasets and LLMs, multitask fine-tuning of a single LLM consistently outperforms single-task models, while also exhibiting greater robustness to noisy labels and stronger cross-domain generalization. To understand these qualitatively different behaviors of LLMs, we conduct a layer-wise representation analysis, showing that cross-task features emerge in middle layers, early layers are highly domain-sensitive, and richer middle-layer representations drive performance improvements, robustness, and generalization. We further show that this mechanism extends beyond AM by observing similar patterns across other tasks. Our findings provide new insights into multitask learning with LLMs, highlight its potential for unified, efficient, and robust language understanding, and motivate layer-wise parameter efficient adaptation strategies within and beyond AM.

More from EMNLP 2026

All EMNLP 2026 papers →