Research
NAIRL’s Professor Tae-Hyun Oh of KAIST to Present Eight Studies, Including Three at NeurIPS 2026
A research team led by Professor Tae-Hyun Oh of the School of Computing at KAIST, affiliated with the National AI Research Lab (NAIRL), is presenting eight studies in the second half of the year. Three papers have been accepted to NeurIPS 2026 and one to EMNLP 2026, one appears in the International Journal of Computer Vision (IJCV), and three appear in IEEE Robotics and Automation Letters (RA-L), two of which will be presented at ICRA 2027. The work ranges from AI that understands space through sound to quality control for video generation, GUI agents, and robot perception in bad weather and darkness.
The first NeurIPS paper, “Hear, Localize, and Reason: Spatially Aware Scene Understanding for Audio-visual LLMs,” asks whether an AI can hear a sound, work out where it is coming from, and connect it to the scene in front of it. The team built HLR-AVSceneQA, a benchmark that tests these three steps in turn, and proposed HLR-LLM, which adds a binaural spatial audio pathway, akin to judging direction with two ears, to an existing audio-visual model. Existing models proved limited at locating sounds and reasoning about their relationships, while HLR-LLM substantially improved spatial understanding without losing its semantic perception. The study was conducted in collaboration with UT Austin.
The second, “Early Failure Detection and Intervention in Video Diffusion Models,” addresses the fact that text-to-video models occasionally produce failures that only become apparent once generation is complete. The team designed a real-time inspection module that converts intermediate results into preview videos and checks them with existing evaluators. The check takes 39.2 milliseconds, negligible next to the 4.3 seconds the same model needs for a single generation step. The pipeline intervenes in stages only when failure is predicted, improving quality with up to 2.64 times less time overhead than regenerating after the fact.
The third, “FacEDiT: Unified Talking Face Editing and Generation via Facial Motion Infilling,” tackles editing a video of someone speaking so that replacing, inserting, or deleting words leaves lip movements and expressions seamless. The team devised a training scheme that masks facial motion over a span of video and reconstructs it from the speech and surrounding context, allowing the model to learn editing without dedicated editing data. They also released FacEDiTBench, the first benchmark for talking head video editing. This study, too, was conducted with UT Austin.
Five further studies follow beyond NeurIPS.
“IGG: A Benchmark for Interactive GUI Grounding under Visibility Constraints,” accepted to EMNLP 2026, tests whether an agent can first reveal a button that is not yet on screen, by scrolling or moving the mouse, and then click it. Humans scored 94.6 percent; GPT-5.5 reached 79.4 percent and the strongest open-source model only 31.6 percent.
“LiDAR-Anchored Collaborative Distillation for Robust 2D Representations,” published in RA-L 2026 and to be presented at ICRA 2027, borrows 3D information from LiDAR to keep camera-based recognition models steady in rain, snow, and darkness. The study was conducted with the autonomous driving company RideFlux.
“DarkQA: Benchmarking Vision-Language Models for Visual-Primitive Question Answering in Low-Light Indoor Scenes,” also in RA-L 2026 and at ICRA 2027, measures how the basic perception of vision-language models breaks down as indoor lighting dims step by step, using 9.4K question-image pairs.
“Sound2Vision: Generating Diverse Visuals from Audio through Cross-Modal Latent Alignment,” in IJCV 2026, generates an image of a scene from everyday sounds alone, in collaboration with UNIST. “DRIP: Self-Improving Robot State Estimation In-the-Wild via Dense Robot-Fitting in the LooP,” in RA-L 2026, proposes a training loop in which a model that estimates a robot’s pose and joint angles from a single image improves its own accuracy in real-world settings.
NAIRL will continue to introduce research that expands how AI sees, hears, and acts to the broader domestic and international AI community.
HLR-LLM project page: https://hlrllm.github.io/
Early Failure Detection: https://arxiv.org/abs/2603.14320
FacEDiT project page: https://facedit.github.io/
IGG project page: https://igg-bench.github.io/
Sound2Vision code: https://github.com/kaist-ami/Sound2Scene
