Skip to content
conftrace_
2026 EMNLP EMNLP 2026

TimlMemBench: A Diagnostic Benchmark for Governed Use of Long-Term Memory in Dialogue Agents

AbstractLong-memory benchmarks usually ask whether an agent can recover evidence from prior dialogue and answer questions about it. Persistent assistants also face governed-use decisions: a remembered fact may later be revoked, superseded, restricted to a private context, or irrelevant to the current request. We introduce TimlMemBench, a diagnostic benchmark for governed use of long-term memory in dialogue agents. It starts from reviewed raw sessions and anchors each derived memory asset back to source evidence. The current package contains 208 reviewed Chinese and English samples, 1,566 sessions, 1,715 questions, and 1,821 governance or activation cases. TimlMemBench reports two complementary evaluations. The canonical-pool LLM-QA control compares systems over the same sample-local memory pool. The raw-native evaluation requires external memory systems to ingest raw sessions before answering governed-memory cases. Experiments show that memory-equipped systems reach high question-answering accuracy in the canonical-pool control, but their raw-native behavior is less stable on lifecycle updates, scope and boundary control, provenance recovery, activation, and restraint. TimlMemBench therefore exposes memory failures that are invisible to retrieval-oriented or answer-accuracy-only evaluation.

More from EMNLP 2026

All EMNLP 2026 papers →