Stop Fine-Tuning to Fix Retrieval Problems — Anant Srivastava

Anant Srivastava, principal technologist for data and AI platforms at Oracle, gave this talk at the AI Engineer World's Fair 2026 in San Francisco. His question is where an AI product's knowledge should live: in the prompt, in memory (retrieval), or in the model's weights. He treats these as three tools for three jobs, and argues against climbing from one to the next whenever answers go wrong.

The opening chapters (0:00 to 2:00) set up the problem. Teams rarely make this choice on purpose; months of ordinary product work make it for them, and every prompt edit, indexed document and training sample is an architecture decision. At 2:40 he describes a support assistant that kept inventing product names after a fine-tune, and at 4:34 he explains how that kind of system piles up by accident.

The middle of the talk gives a diagnostic for each layer. From 5:14, the prompt is for small, stable behavior, with an example support agent prompt at 5:59. Memory follows at 6:59: knowledge that is current, large, citable or access-controlled belongs there. He covers the wrong job for memory (8:09), why access control sits in memory (8:49), and a code assistant example (9:14) with chunking and metadata for code RAG (10:08).

Weights come up at 11:23. His rule is that only reflexes that have stopped changing go into the model. The fine-tuning mistake is at 12:28, cases where fine-tuning works start at 13:53, and a medical coding example at 15:08 shows fine-tuning a reflex. A short section at 16:13 asks whether a problem is about capability or cost.

A decision table at 16:58 pulls the three layers together. From 17:33 he describes a circulating architecture, where knowledge moves between context, memory and weights over time and the agent gets better by doing its job. The final chapter, at 19:27, is titled "The model is the easy part".

The talk suits engineers and tech leads deciding between prompting, RAG and fine-tuning, or anyone who has watched a fine-tune make an assistant less accurate.