Skip to content
conftrace_
2026 EMNLP EMNLP 2026

๐œ†-Diffusion Language Models: Controllable Generalization of Continuous and Discrete Diffusion Dynamics

AbstractDiffusion language models (DLMs) have emerged as a promising alternative to autoregressive language models. While discrete DLMs achieve strong performance across language generation tasks, continuous DLMs remain relatively underexplored despite their smooth latent-space dynamics. In this work, we present a unified embedding-space interpretation of continuous and discrete DLMs. Within this framework, we analyze continuous diffusion dynamics as a retention channel that preserves intermediate denoising information during sampling. Motivated by this perspective, we propose ๐œ†-DLM, which smoothly interpolates between continuous and discrete diffusion dynamics through a controllable parameter ๐œ†. By training with randomly sampled ๐œ†, a single model generalizes across continuous, discrete, and intermediate diffusion dynamics, unifying diverse retention behaviors within one framework. Unlike prior methods that rely on fixed or self-confidence-guided diffusion dynamics, ๐œ†-DLM enables controllable exploitationโ€“exploration trade-offs without retraining. Experiments on LM1B and OWT unconditional generation benchmarks show that ๐œ†-DLM consistently outperforms strong baselines. Moreover, varying ๐œ† enables semantically diverse generation without additional training. Compared with temperature sweeps, varying ๐œ† shows a more structured relationship between lexical diversity and sentence-embedding-based mode coverage.

More from EMNLP 2026

All EMNLP 2026 papers โ†’