Andrej Karpathy – Bigger Models Aren’t the Answer

Andrej Karpathy makes the case that small models, the right tools, and tight feedback loops matter more than raw scale.

For years, the assumption in AI has been simple: bigger model, better agent. More parameters, more compute, more scale.

Andrej Karpathy just made the opposite case.

In a 16-minute breakdown, he argues that raw size isn’t the real bottleneck anymore. What actually determines whether an agent works well is the tools it has access to, and how tight the feedback loop is between taking an action and correcting it.

A small model with the right tools and a fast feedback loop can outperform a much larger model working in isolation. The intelligence isn’t just sitting in the weights — it’s in the system built around the model: what it can call, what it can check, and how quickly it can learn from being wrong.

It’s a reframe worth sitting with, especially for anyone building agents right now. The instinct to scale up may be solving the wrong problem.