In an effort to help AI models learn directly from their missteps, authors Kunlun Zhu, Xuyan Ye, Yibo Li, Cheng Qian, Beibin Li, and Heng Ji have published the Agent Error Dataset (AED) on Hugging Face (one of the leading open-source platform and community repository for machine learning models and research papers). Their newly released collection spans 50,228 error-diagnosis pairs across 9,961 tasks, 33 text-based environments, and 23 policy models, offering developers a standardized way to analyze agent failures and improve post-training efficiency without needing to re-run expensive rollouts from scratch.
The researchers developed a five-stage Agentic Error-to-Training pipeline that collects natural agent failures, generates diagnostic feedback, and validates proposed action corrections against recorded evidence. In matched replay evaluations across 3,062 test cases, initial corrective proposals increased verifier pass rates from 18.4% to 51.1%.
Furthermore, fine-tuning the Qwen3-8B (Qwen3 is the latest generation of LLMs in Qwen series) model on the dataset’s diagnostic data raised its step-by-step diagnostic agreement with teacher labels from 47.2% to 63.6%, while action-repair training outperformed traditional success-only training methods on standard benchmarks.
