Bending Scaling Laws with Brighter Algorithms

Scaling laws suggest the intuitive proposition that “more resources = higher performance” — that AI performance improves by brute-force scaling of data and compute. In practice, however, this approach is losing sustainability, making algorithmic innovation that maximizes computational efficiency essential to move beyond scaling laws.

This talk introduces recent work in that direction: gradient-based methods for improving synthetic data; symbolic search algorithms for test-time reasoning; test-time training that extracts additional learning during inference; and new tokenization algorithms that enable more accurate and faster inference.

This seminar and a speaker interview were featured as the “NAIRL Insight Talk” special in NAIRL Frontier Letter Vol.01 (Aug 2025).