Improving the performance of AI models typically requires larger models, more data, and greater computational resources. This phenomenon is described by the Neural Scaling Law, which states that model performance increases along a predictable curve as scale grows. However, this also comes with exponential increases in cost and resource consumption, creating fundamental limitations.
The research conducted by the National AI Research Hub aims to overcome these limitations and develop sustainable AI technologies that achieve both resource efficiency and high performance.

This project—jointly conducted by KAIST, Korea University, Yonsei University, and POSTECH—proceeds across four major subfields. First, to reduce computation without sacrificing performance in large-scale models, the team is developing lightweight Korean-specialized language models and advanced knowledge distillation techniques. In particular, methods such as prompt distillation, training-free multi-criteria token merging, and mixed-bit quantization are being utilized to minimize inference-time computation while preserving information.
Additionally, the team is developing techniques to analyze the internal knowledge learned by generative models—such as diffusion models—and transfer it into smaller, more efficient models.

Second, the project aims to relax the constraints of scaling laws through new training algorithms. By applying Learning to Optimize (L2O)–based model update methods, reinforcement-learning–driven computation resource allocation strategies, and data refinement and compression techniques grounded in the information content of training data, the team is working to accelerate training while reducing unnecessary data and computation. In particular, multi-agent cooperative algorithms and differential privacy techniques applicable to high-resolution generative models are expected to have significant impact in future real-world applications.

Third, the project focuses on developing new model architectures to overcome the structural limitations of neural scaling. This includes improving the self-attention mechanism through graph signal processing, and designing efficient multimodal architectures that reduce the computational complexity of conventional Transformers from O(N²) to O(N)—allowing the model to maintain strong performance on long-sequence inputs while reducing computation. The team is also developing object-centric representation learning for unsupervised semantic segmentation and speculative decoding strategies to improve inference efficiency.