ParaShard: Knowledge Sharding for Effective Parametric Retrieval-Augmented Generation
Abstract
AbstractRetrieval-Augmented Generation (RAG) enhances large language models with external knowledge, but long retrieved contexts incur high computational cost and often degrade performance due to context–knowledge mismatch. Parametric RAG (PRAG) addresses this by integrating external knowledge into model parameters via lightweight adapters. However, existing PRAG methods are fundamentally document-centric, treating each document as an independent unit of knowledge, which leads to poor scalability and weak supervision. We argue that knowledge is inherently distributed across semantically related documents, and document-level parameterization is a suboptimal granularity. To this end, we propose ParaShard, a knowledge-centric framework that organizes knowledge into semantically coherent knowledge shards and parameterizes each shard with a shared adapter. This design enables richer supervision across related documents while significantly reducing the number of required adapters. At inference time, ParaShard dynamically selects and composes multiple shards using a diversity-aware strategy to ensure comprehensive knowledge coverage. Experiments on four knowledge-intensive QA benchmarks show that ParaShard achieves superior average performance across diverse knowledge-intensive tasks compared with strong RAG and PRAG baselines, with particularly strong gains on multi-hop reasoning, and substantially reduces storage cost compared to PRAG, demonstrating the knowledge injection effectiveness and storage efficiency of knowledge-centric parameterization.