Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Uncertainty-Aware Hybrid Retrieval for Long-Document RAG

AbstractRetrieval-augmented generation (RAG) depends critically on the quality and granularity of retrieved evidence. Large retrieval units preserve context but often introduce irrelevant content, which can dilute answer-bearing evidence and worsen long context utilization. Fine-grained units are more compact, but they may be difficult to retrieve reliably because short chunks can lack semantic, lexical, or bridging cues needed to match the query. We propose **Uncertainty-aware Multi-Granularity RAG (UMG-RAG)**, a training-free hybrid retrieval framework that adapts retrieval across chunk granularities based on query-specific reliability estimates. Instead of training a new retriever or modifying the generator, UMG-RAG uses existing dense and sparse retrievers as complementary experts across multiple chunk granularities. For each query, it converts each expert-granularity score list into an evidence distribution, estimates reliability from distribution entropy, and fuses candidates according to query-specific semantic, lexical, and granularity confidence. We further introduce **UMGP-RAG**, a parent promotion variant that uses fine-grained hits to locate relevant evidence while returning broader non-redundant parent chunks for local coherence. Experiments on question answering benchmarks show that uncertainty-aware fusion and parent promotion improve generation quality in a training-free, plug-and-play retrieval pipeline.

More from EMNLP 2026

All EMNLP 2026 papers →