Skip to content
conftrace_
2026 EMNLP EMNLP 2026

RAG Components Are Not Independently Rankable

AbstractRAG systems are usually compared as complete pipelines, making it difficult to attribute improvements to individual components. We evaluate chunking, retrieval, fusion, and reranking through a full factorial design with paired component substitutions and equal context budgets. Across five benchmarks, chunking interacts with retrieval architecture, BM25 fusion changes direction by domain, and reranking is the only intervention we test that remains beneficial across nearly all upstream configurations. Claim-level transitions further show that similar net recall can conceal very different changes in visible evidence. These findings demonstrate that RAG components cannot be ranked independently and motivate interaction-aware evaluation instead of winner-take-all pipeline comparisons.

More from EMNLP 2026

All EMNLP 2026 papers →