Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Training-Free Generative Retrieval

AbstractLarge language models (LLMs) are emerging as versatile foundations for generative retrieval (GR), a paradigm that directly generates relevant document identifiers (docids) given a query. Existing ways of using LLMs for GR typically rely on heavy post-hoc training to fit a relevance data prior, often leading to overfitting and poor cross-domain generalization. Can retrieval, like other language tasks, be performed without retriever-specific training using LLMs? We conduct a preliminary study and find that constrained decoding, required to ensure valid docids, often misaligns with the model’s natural free-generation patterns. We propose FreeGR, a plug-and-play framework that adapts retrieval data to a LLM’s habitual generation behavior. FreeGR consists of three components: an LLM self-annotated docid generator, an LLM-preferred input context designer, and two decoding strategies that trade off retrieval effectiveness and efficiency. It requires no retriever-specific parameter optimization, is model-agnostic, and scales across LLM families and sizes while preserving their generative capabilities. Experiments on the NQ, MS MARCO, and BEIR datasets show that FreeGR achieves competitive retrieval effectiveness compared with several trained generative and dense retrievers, without retriever-specific parameter optimization.

More from EMNLP 2026

All EMNLP 2026 papers →