Skip to content
conftrace_
2026 AACL AACL 2026

Remember, Verify, or Ask? Evaluating Memory Decisions at Reuse Time

AbstractPersistent memory can personalize an LLM agent, but an incorrect durable update can distort future behavior. We introduce the Memory–Clarification Boundary benchmark (MCB), which supplies an interaction-derived candidate and a later reuse context and asks whether to persist it, use it locally, verify it, or clarify it with the user. The release contains 140 primary scenarios, 70 evidence-flip contrasts, and a 36-item real-dialogue-derived extension. On 70 adjudicated primary-test items, verification recall exceeds clarification recall in all 12 label conditions across four checkpoints. However, only three comparisons remain significant after an exploratory Fisher test with Holm correction, and none involve Claude. A category oracle attains .800 accuracy, exposing strong scenario-category structure. Original label and tool-selection conditions agree on only 23–57% of items, although their differing prompts prevent attributing this solely to the interface. The naturalistic extension is descriptive: it contains only two clarification items and evaluates local models only. MCB measures reuse-time decision selection under a conservative policy; storage backends, matched policy/tool interventions, and downstream utility remain unevaluated.

More from AACL 2026

All AACL 2026 papers →