Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Knowing but Not Doing: Discourse-level Constraints Adherence of Large Language Models

AbstractInstruction following in large language models (LLMs) is commonly evaluated at the sentence or surface level, leaving discourse-level constraints underexplored. We study discourse-level instruction following by formalizing it as a structured constraint adherence problem and constructing both explicit and implicit benchmarks for evaluation. Experiments are conducted across models of different scales, revealing a systematic recognition–adherence gap: models can identify discourse constraints but frequently fail to adhere to them during generation with specific failure modes. This failure stems from a fundamental misalignment between standard training objectives and global structural integrity. Importantly, models show substantially improved adherence when discourse-level structural objectives are explicitly optimized.

🌉 Interdisciplinary Bridge - Artificial Intelligence and Natural Language Processing

More from EMNLP 2026

All EMNLP 2026 papers →