Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Trust Before Fusion: QIMG-7 and Source-Aware Resolution for Polluted Multimodal RAG

AbstractMultimodal retrieval-augmented generation (RAG) is usually evaluated with clean evidence, although deployed systems may retrieve topically relevant but unreliable text or images. We introduce QIMG-7, a controlled post-retrieval stress test for multi-sentence factual QA comprising 110 questions from four datasets, seven visual-evidence pollution families, and 16 paired clean/polluted regimes per question (1,760 rows per method). Across four generator/gate stacks, Full-MM is fragile under polluted text: in the main gpt-4o-mini stack, support decreases from 0.908 with clean text to 0.490 with polluted text. We propose source-aware trust resolution (SATR), a training-free family that compares Parametric, Text-only, and Full-MM candidate answers and selectively chooses a branch or falls back. Field-Selector obtains the highest numerical balanced score, 0.816, improving over Full-MM by 11.7 points and the Cascaded Router by 2.2 points. TextTrust, an image-blind text-reliability baseline, reaches 0.800; Field-Selector’s 1.5-point balanced advantage over it is not statistically resolved (95% CI [-0.006, 0.036]). In this text-first setting, the results support selective trust rather than unconditional multimodal fusion. Artifacts are available at https://github.com/mbzuai-nlp/Trust_Before_Fusion.

More from EMNLP 2026

All EMNLP 2026 papers →