Information Retrieval: What Really Matters in Agentic Search?

Large language models and agents can now answer remarkably complex questions, yet without the right information, no amount of reasoning is useful. Jimmy Lin, Professor at the University of Waterloo, who has worked on information retrieval (IR) for nearly three decades, argued that search has become even more important in the age of agents, and pointed to fully open models as the path through which academic research can contribute.

On October 13, the National AI Research Lab (NAIRL) and the KAIST Kim Jaechul Graduate School of AI co-hosted a monthly Distinguished Scholar Seminar at the Seoul AI Hub, featuring Lin as the invited speaker. The hybrid event drew around 320 participants in total, with approximately 150 attending on-site and 170 joining online.

Jimmy Lin is the David R. Cheriton Chair at the University of Waterloo and one of the leading researchers in information retrieval and natural language processing. As an international participating professor in NAIRL’s joint research program, he has worked closely with NAIRL researchers.

The talk was built around three questions: where search is heading, what is left for researchers to do, and where academic research can make a real contribution.

To answer the first question, Lin traced the history of search from clay tablets and scrolls through libraries to web search. The principles of the vector space model, proposed more than 50 years ago, are still in use today. Keyword search gave way to transformer-based vector search and reranking, and around 2023 came retrieval-augmented generation (RAG), which feeds search results to language models. The next step was to break components such as lexical search, vector search, and reranking into individual “tools” and let models learn, through reinforcement learning, how to use them. Defining agents simply as “models using tools in a loop,” Lin explained that agentic search is moving toward taking over query formulation, task planning, and synthesis of results, work that users once did themselves.

His answer to the second question was simple. Whether on the web, in enterprise data, or in email, the relevant information still has to be found, and if an agent cannot find it, no amount of harness or context engineering will produce the right answer. Search, he stressed, matters even more in the agentic world, and the problem is far from solved.

For the third question, Lin put forward “rigorous disentangled understanding” as the key idea. Today, no one can say precisely whether gains in the latest models come from better tools, better models, or better harnesses. Citing Richard Feynman’s “What I cannot create, I do not understand,” he argued that disentangling these effects requires fully open models, with data, code, and configurations all available and full control over every upstream step. Open-weight models alone are not enough, he noted, because there is no way to know what data they were trained on or how.

Lin also presented recent work that puts this view into practice. His team pretrained a backbone model of about three billion parameters from scratch on openly available raw corpora, then fine-tuned it on public datasets to build a reranker. The model performed on par with a roughly four-billion-parameter model built on an existing open-weight backbone, with pretraining done on a single GPU server and fine-tuning completed in about a day on two H200s. The result shows that by focusing on a specific domain, it is possible to build a model with end-to-end control using academic-scale resources. He added that controlling the training data also makes it possible to tell whether a model answers by memorization or by actual reasoning.

On October 12, the day before the seminar, Lin held one-on-one research meetings at NAIRL with seven participants, including students from the KAIST Kim Jaechul Graduate School of AI and a NAIRL postdoctoral researcher. Participants discussed their ongoing research topics and directions in depth and received his advice on recent research trends and future directions.

NAIRL will continue to broaden exchanges with leading researchers worldwide, strengthening its role in connecting Korea’s research community with the global AI research network.