Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Capture Is Easy, Forgetting Is Hard: Benchmarking Prospective Memory in Conversational Agents

AbstractConversational agents are good at recall. But recall only reacts and an agent that answers only what it is asked can never raise what the user never thought to mention. We study the other direction which we call prospective memory. A background agent keeps a small set of pins in the assistant’s context before any question arrives and each pin must be retired by the right authority. A constraint from chat, such as a stolen bike, ends only on the user’s word. One from a city feed, such as a suspended tram line, ends only on the feed’s end message, whatever its forecast said. We release a benchmark of 60 days, ten users and 1,802 graded decisions in a mobility assistant, with automatic answer keys for pinning, forgetting, recommendation and habit discovery. Capture, at least of the templated reveals this benchmark renders, is the easy half. A dense 8B model captures events almost as well as a 397B/17B sparse one and what separates the two is forgetting. The small model hoards and invents. The large model’s one systematic failure is a release cascade, asked to rewrite its whole pin table, it treats any release signal as a reason to shorten it. Asked instead to name its removals as explicit operations, it names only the right ones, every swept pin survives, and required session recall rises from 0.77 to 0.99, from 0.80 to 1.00 on the events both runs captured.

More from EMNLP 2026

All EMNLP 2026 papers →