Skip to content
conftrace_
2026 EMNLP EMNLP 2026

CoopAgent: Training Multi-Agent Cooperation from Single-Agent Execution Trajectories

AbstractLLM-based multi-agent systems have demonstrated strong potential for solving complex tasks, yet reliable collaboration remains challenging. While most training-based methods rely on multi-agent trajectories, we investigate whether cooperative behaviors can be strengthened using only single-agent execution data. We propose CoopAgent, a training framework that learns solely from single-agent execution trajectories and transfers to multi-agent deployment. To bridge the gap between single-agent training and multi-agent environments, we introduce PADPO (Perturbation-Aligned DPO), which aligns action preferences between original observations and task-related perturbations that mimic changes caused by other agents. We train CoopAgent on single-agent execution trajectories collected in Minecraft and evaluate it on VillagerBench. The results demonstrate its potential to enhance collaboration on VillagerBench while preserving strong single-agent execution ability. We further assess CoopAgent on out-of-domain benchmarks, showing improvements in several multi-agent settings and performance comparable to the multi-agent training method AT-GRPO on the coding benchmarks. Our code is available at https://github.com/hlpzz/CoopAgent.

More from EMNLP 2026

All EMNLP 2026 papers →