Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Same Evidence, Different Decisions: Document Segmentation Bias in Retrieval-Augmented Generation

AbstractDocument chunking is widely treated as neutral preprocessing in retrieval-augmented generation, but in content-balanced interventions, re-segmenting the same evidence, without changing its semantic content, can shift fixed-choice LLM decisions and flip them in near-threshold regimes. Across full ratio-sweep experiments on 9 primary queries and 6 LLMs, document-unit structure systematically shifts model choices. We fit a shifted sigmoid whose steepness captures structural sensitivity and whose offset c captures fitted decision asymmetry. The two fitted parameters enable post-hoc flip-point estimation after per-query calibration, not a priori prediction. The shifted-sigmoid fit reaches R² ≥ 0.80 in 38/41 non-degenerate model–query combinations; the English experiments show the same form on 9 queries and 4 LLMs. It persists under noisy real-corpus retrieval when retrieved evidence is stance-balanced (ΔP = 0.53, p < .001, on one THUCNews retrieval query). The effect is regime-dependent: near-threshold decisions are vulnerable, while large decision offsets resist manipulation. Chunking and boundary placement can therefore affect downstream fixed-choice RAG decisions, with implications for retrieval system design and adversarial robustness.

More from EMNLP 2026

All EMNLP 2026 papers →