Skip to content
conftrace_
2026 EMNLP EMNLP 2026

ClinTraceBench: Source-Verifiable Longitudinal Clinical Reasoning over EHR-Derived Dialogues

AbstractClinical LLM assistants must reason over multi-visit patient trajectories, yet whether the compact history representations used to scale them—retrieval, structured timelines, LLM summaries, agentic memory—preserve the longitudinal signal clinical reasoning needs has not been measured. We introduce ClinTraceBench: 385 MIMIC-IV-derived verified dialogues with event-ID provenance, a nine-task taxonomy (T1–T9), and L0–L4 deterministic + L5 human-audit validation (98.92% agreement). We evaluate eight history representation strategies—a no-context floor, last-visit-only, full-context, BGE-M3 dense-retrieval, two compression schemes, and two agentic-memory systems (Mem0, A-Mem)—across four backbones (DeepSeek-V3, GPT-4o-mini, Haiku 4.5, Sonnet 4.6) on 6,271 questions: 32 cells, 200,672 predictions. Four findings emerge: (F1) compressed strategies pay an aggregation tax on multi-visit trends and cross-patient comparisons; (F2) the blind-to-full gap spans +29.8 percentage points (GPT-4o-mini) to +62.7 (Haiku); (F3) abstention scales non-monotonically with context length; (F4) a controlled T3 injection probe isolates compression-induced relation loss—with the attribution sentence present before construction, Mem0, A-Mem, and llm-summary still recover only 0–5.3% of the injected positives. On the Pareto frontier Haiku dominates Sonnet under full-context (25.76 vs. 106.21), inverting the “biggest backbone wins” heuristic.

More from EMNLP 2026

All EMNLP 2026 papers →