Post-Deployment Self-Improving Language Agents: Learning, Memory, and Adaptation
Abstract
AbstractLLM-based systems are increasingly deployed as language agents: long-running NLP systems that retrieve information, call tools, maintain memory, interact with users, and operate in changing environments. After deployment, these systems must contend with evolving user goals, updated knowledge sources, shifting APIs, new tasks, and previously unseen failure modes. Yet adaptation is still often handled through an ad hoc combination of prompt changes, periodic fine-tuning, retrieval updates, memory mechanisms, and manual intervention. The tutorial examines how post-deployment NLP systems can instead be designed as self-improving language agents, drawing on never-ending learning, lifelong learning, continual learning, and recent advances in agentic AI. We take a system-level view of learning: improvement may arise not only from changes to model parameters, but also from persistent changes to memory, knowledge, tools, policies, and other components of the agent and its harness. The tutorial combines literature review with practical running examples (code provided) illustrating different forms of post-deployment learning. We’ll examine empirical results and failure cases to understand when adaptation improves a system, when it doesn’t, and how such changes can be evaluated. We’ll cover a broader transition in NLP: from increasingly capable static models toward developing language systems that can continue to learn and improve responsibly throughout their deployed lifetime.