Fable 5 Tops the NanoGPT Speedrun: 153 Autonomous Runs, 18 Models, and Who's Really Closing the Gap on Humans
Fable 5 Tops the NanoGPT Speedrun: 153 Autonomous Runs, 18 Models, and Who's Really Closing the Gap on Humans
On August 22, Prime Intellect dropped the NanoGPT Speedrun Frontier leaderboard — 18 frontier models, each given a coding agent and 24 hours to squeeze nanoGPT's training loss as low as possible.
The result goes against the grain: the leader isn't OpenAI, isn't Anthropic. It's Fable 5, which closed 81.7% of the gap to the human record.
"We ran 153 autonomous runs across 18 frontier models on the nanoGPT optimizer speedrun."
That's the original line — and I checked every number below against it.
What this benchmark actually measures
The NanoGPT Speedrun isn't about how fast a model writes code. It's about autonomous research capability: hand an agent a nanoGPT optimization task, let it read code, tweak hyperparameters, run experiments, iterate — and see how close it gets to the human expert record of 2,600 steps within a fixed budget.
Prime Intellect ran each model multiple times (153 runs total) and took the best validated result. The harness behind each model is public too — claude-code, codex, kimi-code, qwen-code, the works.
This isn't a traditional benchmark. It tests how far a model can go when you drop it into an open-ended optimization problem and walk away.
Reading the leaderboard: the gaps are bigger than you'd think
| Rank | Model | Best Record | Gap Closed | Harness |
|---|---|---|---|---|
| 1 | Fable 5 | 2,726 | 81.7% | claude-code · high |
| 2 | Opus 5 | 2,920 | 53.6% | claude-code · max |
| 3 | Kimi K3 | 2,930 | 52.2% | prime-agent · max |
| 4 | Kimi K3 | 2,974 | 45.8% | kimi-code · max |
| 5 | Opus 4.8 | 3,018 | 39.4% | claude-code · max |
| 6 | GPT-5.6 Sol | 3,042 | 35.9% | codex · xhigh |
| ... | DeepSeek V4 Pro | 3,205 | 12.3% | claude-code · max |
A few things jump out:
- The gap between #1 and #2 is 28 points. Fable 5 (2,726) to Opus 5 (2,920) is 194 steps — more than the distance between several pairs further down the table.
- Same model, different harness, different rank. Kimi K3 hits 2,930 on prime-agent but drops to 2,974 on kimi-code. Your agent framework alone is worth 1-2 positions.
- DeepSeek V4 Pro sits at #13 with 12.3% gap closed — in just 1.1 days and 309 calls. One of the most resource-efficient entries on the board. Efficiency is a different axis entirely.
- Human record: 2,600 steps. Baseline (no optimization): 3,290. Even the winner hasn't touched human level yet — it's 81.7% of the way there.
An analogy: this is racing, not just engine dynos
Think of the model as the engine and the harness as the gearbox and suspension. Same engine, different gearbox, and lap times change dramatically. Fable 5 went fast partly because claude-code — its "gearbox" — was tuned to burn tokens on trial and error: 800M total tokens, 811 experiments, the most spendthrift entry on the board.
But here's the twist: spending more doesn't mean running better. GPT-5.6 Sol burned 2.9B tokens and 28k calls and still finished sixth. DeepSeek V4 Pro used 26M tokens to close 12.3% — two orders of magnitude better cost efficiency.
The counterargument: does this benchmark measure real research?
Fair pushback: nanoGPT optimization is a convergent engineering problem with a clear target. Tuning hyperparameters isn't the same as open-ended scientific discovery. True — this board measures "engineering autonomy," not curiosity-driven research.
But here's why it matters anyway: it's the first time anyone has compared agentic models from every major lab under one protocol, one budget, with fully open traces. Before, every lab had its own benchmark and its own story. Now there's a reproducible reference point. Prime Intellect even publishes 41 curated agent trajectories — tool calls, subagents, scratchpads — so you can go read what Fable 5 actually did.
What you can do today
- Head to https://www.primeintellect.ai/research/nanogpt-speedrun and dig through the open traces — see which of Fable 5's 811 experiments paid off.
- If your team runs coding agents, pay attention to harness choice. The board shows the same model moving 1-2 ranks based on the agent framework alone.
- Watch resource efficiency, not just absolute scores: DeepSeek V4 Pro proves you can get usable results on a tiny budget.
Go pull up a trace or two — the scratchpads alone are worth the click.
FAQ
Q: When was the NanoGPT Speedrun Frontier released? A: August 22, 2026, by Prime Intellect — the latest edition of its NanoGPT Speedrun series.
Q: Who is Fable 5, the winner? A: The model that topped the board with a best record of 2,726 steps, closing 81.7% of the gap to the human record via the claude-code · high harness — 811 experiments and 800M total tokens.
Q: Does this mean agents have surpassed humans? A: No. The human record is 2,600 steps; Fable 5's 2,726 is still 126 steps short. Even the winner is only 81.7% of the way there, and tail entries (Kimi K2.7, GPT-5.5) closed just 7-8%.
✨ 本文由 DeepSeek 生成初稿，Claude 审核润色。
Sources:
- Prime Intellect: NanoGPT Speedrun Frontier