VideoNav: Cost-Aware Index Navigation for Long Video Understanding
Abstract
AbstractThe rising demand for long-video understanding presents significant challenges due to massive information and complex temporal dependencies. Direct processing with Large Visual Language Models (VLMs) incurs high computational costs and risks information loss due to frame sampling. Meanwhile, existing agent-based methods often struggle with fragmented contexts and the high overhead of iterative processing. Therefore, we propose VideoNav, which introduces a new paradigm of index-centric navigation, reformulating video understanding as cost-aware structured query processing. VideoNav decouples heavy perception from online reasoning, including offline index construction and online adaptive navigation. In the offline phase, we transform unstructured video data into a structured, multi-granular index tree. During the online phase, the framework executes a top-down, coarse-to-fine navigating strategy. Leveraging the LLM agents, it efficiently locates relevant information and answers the query with navigating guidance, multi-level navigation, dual-path routing and visual inspection. Crucially, this adaptive mechanism enables early termination when high-level information suffices, minimizing redundant computation. Extensive experiments across different benchmarks demonstrate the effectiveness of our framework. VideoNav achieves decent accuracy while delivering 2.5x-5.7x cost savings and a 2.9x-5.8x speedup.