Skip to content
conftrace_
2026 EMNLP EMNLP 2026

OP-Bench: Benchmarking Over-Personalization for Memory-Augmented Personalized Conversational Agents

AbstractMemory-augmented conversational agents enable personalized interactions using long-term user memory and have gained substantial traction. However, existing benchmarks primarily focus on whether agents can recall and apply user information, while overlooking whether such personalization is used appropriately. In fact, agents may overuse personal information, producing responses that feel forced, intrusive, or socially inappropriate to users. We refer to this issue as **over-personalization**. In this work, we formalize over-personalization into three types: Irrelevance, Repetition, and Sycophancy, and introduce **OP-Bench**, a benchmark of 1,700 verified instances constructed from long-horizon dialogue histories. Using OP-Bench, we evaluate multiple large language models and memory-augmentation methods, and find that over-personalization is widespread when memory is introduced. Further analysis reveals that agents tend to retrieve and over-attend to user memories even when unnecessary. We further evaluate existing post-processing strategies for mitigating this issue, but find that they provide only limited improvements and often reduce personalization performance. Our results highlight over-personalization as a challenging and largely unresolved problem in memory-augmented dialogue systems.

More from EMNLP 2026

All EMNLP 2026 papers →