From Decoding Knobs to Commit Frontiers: Understanding Trajectory Control in Discrete Diffusion Language Models
Abstract
AbstractDiscrete diffusion language models generate text by iteratively committing masked positions, making decoding a trajectory-level process rather than a sequence of independent token decisions. Existing analyses often focus on surface-level knobs such as temperature, step budget, block size, or scheduler choice, but these knobs do not explain why the same change can induce different behaviors across models. We analyze discrete diffusion decoding through high-influence commitment events and define an observed trajectory influence measure based on downstream entropy reduction over unresolved positions. Across Dream-v0-Instruct-7B and LLaDA-8B-Instruct on reasoning, code, and question-answering tasks, we find that influence is sparse, early relative to each model’s commit frontier, and stable under tested seed and step-budget variations. We further show that criticality emerges from the interaction between commit timing and token role, explaining why decoding knobs reshape trajectories in model-dependent ways. Finally, lightweight margin-based interventions validate the analysis by correcting diagnosed failure modes without additional forward passes.