The Role of Entropy in Visual Grounding: Analysis and Optimization
Abstract
AbstractRecent advances in reinforcement learning–based fine-tuning of multimodal large language models (MLLMs) have highlighted the importance of entropy control for stable optimization. While entropy has been extensively studied in reasoning-centric settings, its behavior and optimization role in perception-oriented tasks remain insufficiently understood. In this work, we reveal that entropy exhibits fundamentally different roles in perception tasks compared to reasoning, particularly under continuous prediction and reward. Using visual grounding as a case study, we show that entropy can remain misaligned with optimization objectives due to persistent low-probability yet high-advantage samples. Building on these findings, we introduce ECVGPO (Entropy Control Visual Grounding Policy Optimization), an interpretable algorithm designed for effective entropy regulation. Through entropy control, the trade-off between exploration and exploitation is better balanced. Experiments show that ECVGPO achieves broad improvements across various benchmarks and models.