GR2: Mitigating Granularity Collapse via Adaptive Query Routing in Generative Retrieval
Abstract
AbstractGenerative Retrieval (GR) aims to directly generate document identifiers (DocIDs) without external indices. We identify granularity collapse: during training, GR models develop a preference for generating DocIDs with a specific granularity, causing retrieval failures when it differs from that of the target document’s DocID. We propose to quantify this mismatch using "isotropy" and confirm that retrieval accuracy degrades as such granularity mismatch increases. To address this, we propose Granularity Routing for Generative Retrieval (GR2), which trains multiple expert models specialized for different granularity levels and routes each query to the appropriate expert. Experiments on NQ320k demonstrate that improves retrieval performance, with larger gains on queries that suffer more from granularity collapse.