Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Attention-Aware Remasking: A Training-Free Approach for Enhancing Visual Fidelity in Multimodal Diffusion Language Models

AbstractMultimodal diffusion Large Language Models (dMLLMs) emerge as a compelling alternative to traditional autoregressive paradigms. But they still suffer from hallucinations driven by an inherent language bias. Through an analysis of the dMLLM inference dynamics, we observe that visual tokens frequently receive disproportionately low attention scores. We identify that the conventional low-confidence remasking strategy redundantly leverages logit-based probabilities during the token selection process, thereby exacerbating this language bias. To address this vulnerability, we propose Attention-Aware Remasking, a training-free, plug-and-play strategy that incorporates newly defined visual confidence scores to select the final output tokens, effectively promoting visual grounding. To enhance the logical integrity and inference efficiency of dLLMs, we further propose clue-based answering and echo-filling. Evaluations on representative Visual-Language Natural Language Explanation (VL-NLE) tasks, including VQA-X and e-SNLI-VE, demonstrate that our method achieves the highest BERTScore on both VQA-X and e-SNLI-VE, and the best answer accuracy on VQA-X among the compared methods. Our study reveals that the widely adopted logit-based remasking is not necessarily the optimal approach, highlighting promising new avenues for visually grounded inference in dMLLMs. Our code is available at: https://github.com/yechancha/Attention-aware-Remasking.

More from EMNLP 2026

All EMNLP 2026 papers →