The Father of Reinforcement Learning Left John Carmack's Lab to Build AI With No Training Run

TL;DR

Richard Sutton, who won the 2024 Turing Award for co-inventing reinforcement learning, has left John Carmack's AI startup to launch Oak Lab, a Canadian company built on a blunt premise: today's large language models are a dead end, and the way forward is agents that learn continuously from their own experience, with no separate training run. The move was first reported by The Logic. There is no model, no benchmark, and no disclosed funding yet. What there is, is the most credentialed contrarian in the field putting his name on the opposite of the scaling playbook.

What actually happened

Sutton co-founded Oak Lab with Khurram Javed, a former PhD student. Both were at Keen Technologies, the Dallas AGI startup founded by Doom and Quake co-creator John Carmack, where Sutton had been a research scientist since 2024. They left. Oak Lab is incorporated in Canada, and its stated goal, per The Logic's reporting, is agents that "learn in real time without storing or replaying data," improving continuously while using less compute than the current playbook demands.

To see why this is news and not just another lab launch, you need to know who Sutton is. He is, with Andrew Barto, the person the field credits with the conceptual and algorithmic foundations of reinforcement learning: temporal-difference learning (1988), the standard textbook (Reinforcement Learning: An Introduction, 1998 and 2018), and the reward hypothesis that underpins how we frame agents at all. When he calls the current approach "weak and inefficient," that is not a hot take from the cheap seats.

The bet: learn at runtime, not in a training run

Here is the disagreement, stripped of jargon. A modern LLM is trained once, on a huge pile of text, then its weights are frozen and shipped. After that it does not learn. It can look things up and hold a conversation in context, but the model itself is a snapshot. Fine-tuning makes a new snapshot; it does not make the thing learn on the job.

Sutton's bet is that intelligence has to be the other thing: an agent that keeps learning during runtime, from the consequences of its own actions, forever, with no line between "training" and "deployment." (The homely version: an LLM crams for one enormous final exam, graduates, and then answers every question for the rest of its life from that frozen snapshot of what it knew on graduation day. Sutton wants a thing that keeps getting better on the job the way you actually improve at anything, by doing it and noticing what happened.)

What "OaK" actually means

The architecture behind the company is OaK, which Sutton laid out in a 2025 talk and which stands for Options and Knowledge. An "option" is a temporally extended skill: a little sub-policy plus a rule for when it stops (think "walk to the door" rather than one muscle twitch). "Knowledge" is what the agent learns by running those options and predicting what happens next.

The design has three commitments, and they are the whole personality of the thing:

• Domain-general. No world-specific parts hand-built in. The same meta-methods should work anywhere.
• Experiential. Everything grows out of the agent's own runtime experience, not a dataset scraped in advance.
• Open-ended. The agent stacks ever more abstract features, skills, and world-models on top of each other, limited only by compute.

Sutton's own line from the talk is the cleanest summary of the philosophy: "We want AI agents that can discover like we can, not which contain what we have discovered." An LLM, in his framing, is a magnificent container of what humanity has already written down. That is exactly the part he thinks is the trap.

The Bitter Lesson, cited against itself

Now the irony a builder will enjoy. Sutton also wrote "The Bitter Lesson" (2019), the single most-quoted essay of the scaling era. Its thesis: general methods that ride more compute beat clever hand-built human knowledge, every time, by a lot. The GPT camp has treated it as scripture for pointing ever more compute at ever bigger transformers.

Sutton's own reading runs closer to the opposite. To him the lesson was never "scale up pretraining." It was "stop baking in human knowledge and let the machine learn for itself, from experience, at runtime." By that light, a giant model pretrained to imitate a frozen snapshot of human text is the human-knowledge shortcut, not the escape from it. The man who wrote the verse everyone quotes just started a company to build against the thing they think the verse blesses.

Why a builder should care (and what to discount)

Discount first. There is no product, no weights, no benchmark, and no announced raise. This is a research company betting on an unsolved problem. Continual learning has a decades-long graveyard of hard sub-problems: catastrophic forgetting, the stability-versus-plasticity tradeoff, credit assignment over long horizons. Oak Lab has not shown it cracked any of them. Nobody should re-plan a roadmap around a vision talk.

Care, though, for two reasons. One, capital and talent follow Sutton, and a credible RL-first pole pulls researchers away from "one more trillion tokens" and toward agents that adapt after deployment, which is the exact capability every one of us actually wants from an agent in production. Two, the framing is a useful knife for your own stack: almost everything we call an "agent" today is a frozen model wrapped in retrieval and tools, faking an adaptation it cannot actually do. Sutton is calling that bluff out loud. Whether or not OaK ships, that critique is going to age well.

Key Takeaways

• Richard Sutton, 2024 Turing Award winner and co-inventor of reinforcement learning, left John Carmack's Keen Technologies to co-found Oak Lab with Khurram Javed.
• The premise: today's train-then-freeze LLMs are a dead end; the future is agents that learn continuously at runtime from their own experience.
• The technical basis is Sutton's OaK (Options and Knowledge) architecture: domain-general, experiential, open-ended, with no separate training phase.
• The irony: Sutton wrote "The Bitter Lesson," the essay the scaling camp quotes, and reads it as an argument against imitation-pretrained LLMs, not for them.
• Reality check: no product, no benchmark, no disclosed funding, and continual learning's hardest problems remain open.
• Why it matters: a credible RL-first pole reframes what "agent" should mean, and points at post-deployment adaptation, the thing production agents still fake.

Sources: The Logic, Oak Lab, Rich Sutton: The OaK Architecture (RLC 2025), RLC 2025 talk notes, Richard S. Sutton (Wikipedia)