Skip to content
conftrace_
2026 EMNLP EMNLP 2026

AEL: Evolving Agent Harness in Open-Ended Environments

AbstractLLM Agents Harnesses are hand-designed and stay fixed, so agents accumulate experience but never learn how to use it: which memories to retrieve, when retrieved evidence is misleading, and when the retrieval strategy itself should change. We introduce Agent Evolving Learning (AEL), a two-timescale framework that evolves the harness, recasting memory use as online policy selection. A fast Thompson Sampling bandit selects among memory-retrieval policies episode by episode, while slow LLM reflection follows a diagnose-before-prescribe principle: it first explains why performance degraded, then injects a targeted new retrieval policy as a bandit arm when the current pool plateaus. AEL outperforms ten self-improving and nonLLM baselines on a sequential portfolio benchmark, lifting Sharpe by 27% over the strongest memory-only variant with the lowest variance among all stochastic methods, and generalizes to a support-ticket routing stream, where it improves accuracy by 18% over reflectionfree Thompson Sampling and by 51% over the best prior baseline. Mechanism studies further show that the gains are causal: reflection helps precisely when regimes demand different retrieval behavior, and is provably no-harm/nogain when the best policy is stable.

More from EMNLP 2026

All EMNLP 2026 papers →