Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Encoder-only and decoder-only models understand abstract argument structure differently

AbstractLarge language models (LLMs) based on encoder and decoder architectures exhibit strong performance across a range of tasks, yet systematic comparisons of how training objectives shape their behavior remain limited. Building on prior work establishing systematic comparison between decoder-encoder differences (Weller et al., 2026), we ask whether this difference shapes different linguistic bias using two production-related tasks: semantic role assignment and syntactic structure choice (actives vs. passives). Across architectures, LLMs exhibit a robust animacy-first bias, preferring to place animate entities in subject position, diverging from human bias. However, when animacy cues are removed, encoders generalize better than decoders and show a stronger preference for passive constructions. We argue that these differences reflect training objectives: encoders’ sensitivity to contextual compatibility supports flexible mapping from semantics to structure, whereas decoders’ reliance on surface frequency favors dominant patterns and disfavors infrequent structures such as passives.

More from EMNLP 2026

All EMNLP 2026 papers →