NAIRL’s Professor Jinyoung Yeo Team to Present Research on Procedural Compliance and 3D Scene Evaluation at COLM and EMNLP

A research team led by Professor Jinyoung Yeo at the Department of Artificial Intelligence, Yonsei University, affiliated with the National AI Research Lab (NAIRL), will present two research papers at COLM 2026 and EMNLP 2026 on training AI agents to follow operational procedures and on evaluating text-guided 3D scene synthesis. The two studies address how agents can be trained to carry out tasks the way they are supposed to be carried out, and how faithfully generated 3D environments reflect the detailed requirements of a user’s description.

The first study, “Learning to Comply: Workflow-Grounded Environment Generation for Training Procedurally Compliant Agents,” to be presented at COLM 2026, starts from the observation that solving a task is no longer enough for AI agents deployed in real-world settings. Agents must also follow the behavioral policies and operational procedures that govern how a task should be completed, such as verifying a user’s identity or checking a contract before making a change. How to systematically train agents for this kind of procedural compliance has remained underexplored, both in terms of building suitable training environments and in providing learning signals that reward compliant behavior.

To fill this gap, the team, in collaboration with researchers at KAIST, LG AI Research, and Upstage, built a scalable, automated, and verifiable pipeline that generates executable tool-use environments and tasks together, each paired with a ground-truth execution path. The team also compared two ways of rewarding compliant behavior: scoring each individual action as the agent takes it, and evaluating the completed sequence of actions as a whole. Experiments across diverse benchmarks showed that both approaches are effective when the rewards are designed to fit each environment, with step-by-step rewards helping the agent learn which actions matter and whole-sequence rewards providing strong signals about overall compliance. Together, the results establish that scalable environment construction and compliance-aware reward design are both essential for training procedurally grounded agents.

The second study, “Apples on the Table? Evaluating Text-Guided 3D Scene Synthesis via Fine-Grained Constraint Verification,” to be presented at the main conference of EMNLP 2026, tackles the evaluation of 3D scenes synthesized from user-provided text descriptions, a capability crucial for developing embodied agents. Existing evaluation methods either capture only coarse similarity between the synthesized scene and the description, or overlook the spatial reasoning needed to verify object placement, leaving the fine-grained constraints implied by a description, such as “X needs to be in the scene in a Y manner,” unaddressed.

The team, in collaboration with POSTECH, introduced LEGO, a benchmark dataset that pairs each user description with human-annotated constraints and a reference scene, and LEGO-EVAL, an evaluation framework that breaks a description down into small, checkable conditions and verifies each one in turn. A description like “a single room with one bed and two identical armchairs with their backs facing one another,” for example, is split into conditions such as whether the bed is present, whether the two armchairs match, and whether they are placed back to back, and each is checked by locating the objects in the 3D scene and reasoning about their positions. The study shows that LEGO-EVAL detects misalignment far more accurately than existing methods, and that current scene synthesis approaches achieve a success rate of at most 10 percent under LEGO-EVAL, highlighting substantial room for improvement in the field.

These studies are meaningful in that they examine two conditions required for AI agents to operate reliably in real-world settings: following the procedures that govern their actions, and being evaluated against precise, verifiable criteria. NAIRL will continue to introduce research achievements that address agent reliability, evaluation, and real-world applicability to the broader domestic and international AI community.

LEGO-Eval project page: https://gyeomh.github.io/LEGO-Eval/