TICD: Time-Aware Internal Contrastive Decoding for Enhancing Factuality in Diffusion Language Models
Abstract
AbstractDiffusion large language models (dLLMs) generate text through iterative denoising, but they remain prone to factual errors in knowledge-intensive tasks. Through layer-wise and timestep-wise probing, we observe a spatiotemporal stratification pattern in dLLMs: structural-token predictions tend to converge earlier across layers, whereas factual-token predictions rely more heavily on deep layers and become reliable only after sufficient denoising progress. Motivated by this observation, we propose Time-Aware Internal Contrastive Decoding (TICD), a training-free decoding method that calibrates final-layer predictions using entropy-gated intermediate-layer references only during semantically reliable denoising stages. This design uses reliable final-intermediate prediction gaps as correction signals to strengthen factual predictions at the final layer. Experiments on LLaDA-8B and Dream-7B across factuality and reasoning benchmarks show consistent improvements over vanilla decoding.