Skip to content
conftrace_
2026 EMNLP EMNLP 2026

MedRAGChecker: Claim-Level Verification for Biomedical Retrieval-Augmented Generation

AbstractBiomedical retrieval-augmented generation (RAG) produces long-form answers that mix supported, under-evidenced, and contradicted claims, demanding fine-grained verification. We propose MedRAGChecker, a claim-level diagnostic framework that combines textual natural language inference (NLI) with a biomedical knowledge-graph (KG) compatibility signal. Its asymmetric selective-gating rule preserves the textual prediction for most claims and applies KG adjustment only to confident textual entailments assigned low plausibility by the KG back-end. We distill GPT-4.1 claim decompositions and NLI pseudo-labels into compact student models and evaluate three biomedical generators on PubMedQA, MedQuAD, LiveQA, and MedRedQA. The student-ensemble results show that selective BioPortal gating increases macro-averaged faithfulness from 60.8% to 64.4% and reduces the contradiction rate from 19.9% to 15.0%. On 172 annotations of 86 claims whose decisions change after fusion, agreement with human evidence judgments increases from 63.4% to 69.8%. A comparison with DRKG shows that fusion behavior varies substantially with KG coverage, entity linking, score construction, and calibration; the comparison therefore characterizes back-end effects rather than isolating a causal effect of coverage.

More from EMNLP 2026

All EMNLP 2026 papers →