Can Agents Learn from Their Own Failures?

Even the most capable AI agents fail when they meet changing environments, new tasks, or rare real-world cases. Xiaoxiao Li, Associate Professor at the University of British Columbia (UBC) and Faculty Member at the Vector Institute, argued that what matters is not avoiding failure but learning from it. They presented their group’s work on agents that evaluate, diagnose, and repair their own mistakes as a path toward recursive self-improvement.

On October 19, the National AI Research Lab (NAIRL) and the KAIST Kim Jaechul Graduate School of AI co-hosted a special seminar at the Seoul AI Hub, featuring Li as the invited speaker, under the title “Can Agents Learn from Their Own Failures? Lessons from Vision and Medical Tasks for Recursive Self-Improvement.” Around 50 faculty members and graduate students from the KAIST Kim Jaechul Graduate School of AI attended the talk.

Li holds a Canada Research Chair (Tier II) in Responsible AI and a Canada CIFAR AI Chair, and also serves as a visiting researcher at Google. Their visit was organized as part of the partnership between NAIRL and the Vector Institute.

Li began with case studies on vision agents. Retrieval tools (RAG) raise accuracy on average, yet a substantial share of answers flip from right to wrong once retrieved text is added. Sample-level analysis showed that the extra text weakens the model’s attention to the image. With zoom tools, incorrect answers likewise tended to pay less attention to the zoomed-in region. In both cases, their team achieved better performance than existing methods with lightweight, training-free techniques that redistribute attention.

Li then introduced EvalRx, an open-source library that automates this research process. Named after the medical abbreviation for a prescription, EvalRx diagnoses a model, forms hypotheses about why it fails, validates them, and applies a step-by-step prescription ranging from prompt changes to fine-tuning. Every step is recorded in an auditable “health card.”

Finally, Li presented work on optimizing agents’ skills, memory, and tools, noting that giving an agent more tools can actually make it perform worse. They closed by introducing the Vector Institute’s exchange programs, including research internships for students, postdoctoral fellowships, and a visiting researcher program for faculty.

NAIRL will continue to broaden exchanges with leading researchers worldwide, strengthening its role in connecting Korea’s research community with the global AI research network.