Agent-Trident: A Three-Level Progressive Diagnostic Benchmark for Tool-Use Hallucination in LLM Agents
Abstract
AbstractTool-use hallucination has become a critical obstacle to agent deployment. It can emerge and propagate across judgment, planning, and execution. However, existing benchmarks mainly focus on final task completion or individual tool-use capabilities, making fine-grained failure attribution difficult.We propose Agent-Trident (TRIDENT), a three-level progressive diagnostic benchmark for tool-use hallucination. TRIDENT decomposes the agent tool-use process into judgment, planning, and execution, supporting stage-level attribution of tool-use failures. The dataset contains 1,120 samples across 20 subdomains and 7 task structures. We further propose the Cross-level Composite Metric (CCM), a bottleneck-sensitive aggregate metric that measures models’ overall reliability across the tool-use chain. Evaluating 29 mainstream large language models (LLMs), we find that even the best model reaches only 42.79% CCM, indicating that current LLM agents remain far from reliable tool use. Our analysis further identifies model behaviors and stage-level differences overlooked by conventional evaluations. These results show that TRIDENT complements final-outcome evaluation with stage-level diagnosis, revealing reliability limitations in solvability judgment, tool-use planning, and dynamic execution while supporting agent model selection, error analysis, and safe deployment.