CuRLRank: Adaptive Curriculum Reinforcement Learning for Reasoning-Intensive Reranking
Abstract
AbstractDocument ranking aims to order candidate documents based on their relevance to a given query. However, identifying relevant documents in complex real-world scenarios requires deep reasoning beyond surface-level matching, driving a shift from semantic matching to reasoning-intensive reranking. Since the latter requires integrating information across documents for complex reasoning, it presents two key challenges: (1) varying reasoning difficulty, where assessing document relevance to the query requires multi-step reasoning with different complexity; and (2) strong contextual sensitivity, where the relevance of a document is heavily influenced by other documents within the context window. To address these challenges, we propose CuRLRank, which consists of (1) CuRL, an adaptive curriculum reinforcement learning approach that enables LLMs to develop complex reasoning capabilities across varying levels of difficulty, and (2) IFRS, an iterative filtering ranking strategy to achieve efficient and robust ranking. Extensive experiments on BRIGHT and R2MED benchmarks demonstrate that our CuRLRank significantly outperforms baseline methods while achieving lower ranking latency.