Eval Is All You Need
On jagged AGI, solving biology, and world models
“Fleming found that a mysterious antibacterial fungus had grown on a Petri dish he’d forgotten to cover in his laboratory: he discovered penicillin by accident. Scientists have sought to harness the power of chance ever since.” — Peter Thiel, Zero to One
Winter of 2025, I built a cross-domain multi-agent system I called Dual-Use Tech Bridge. While the name isn’t very creative, I think the premise is among one of the last inventions we have left to get right ourselves. I trained hundreds of agents on separate fields, some narrow specialists and others generalists, and set them in discussion with one another in hopes that a connection neither field would be able to make alone would appear out of the conversation. After hitting the limits of my Claude Max plan day after day, I still produced nothing truly novel. The problem couldn’t have been that the models didn’t know enough, as even the models with the fewest parameters have had exposure to more text than any human alive or dead.
May 20, 2026: An internal OpenAI reasoning model disproved the planar unit distance conjecture, an 80-year-old math problem.
My mistake was in assuming memory was the bottleneck. Until last year, a model could only make a forward pass instead of one that pauses or revisits previous steps. In 2025, models started being able to correct themselves with verification programs. Closing the verification loop this way is how we’ve been able to solve over 50 Erdos problems in the past year alone. Math is the first field to show such rapid progress due to the ease of verifiability. Similar to how Fleming found penicillin through pure chance, simulating millions of possibilities and knowing exactly which succeed is how we’ve constructed the vague shape of super-intelligence without actually having it.
Biology isn’t the same type of problem, as it has no simple verifier. While a proof can resolve in a second, the effects of a drug might take a decade inside a body to show up at a detrimental cost to its host, and even then, the answer could be unclear. Additionally, most published biology does not replicate, so there is no stable ground to reason over. AlphaFold was able to succeed because protein structures come with a rare clean dataset and verifiability. Eroom’s Law, which is Moore’s Law backwards, observes that the number of new drugs approved for every billion dollars spent on R&D has halved every nine years since 1950. Simultaneously, we’ve lived through the greatest accumulation of biological knowledge and compute in history. Dario Amodei’s framework for this misconception is the marginal return to intelligence. In Machines of Loving Grace, Amodei argues that intelligence isn’t the constraint so much as the physical world is.
The next frontier is manufactured verifiability. The most apparent solution in biology would be a simulator. The virtual cell is a model that is supposed to predict how a cell answers a perturbation, so that a million hypotheses can be made and quickly tested. If we scale this up, Fei-Fei Li’s bet on world models are relatively the same idea. Her claim is that text will never capture the nuances of the physical world, and spatial intelligence will be able to solve problems LLMs structurally cannot. Much like how Leonardo drew the aerial screw four centuries before there were engines to turn it, we are now able to render the machine while having no clear idea of how to build it. The problem with a truly trustworthy cell simulator is that the needed verification is still in the physical world, and we are not currently able to map that world finely.
Peter Thiel mentions in Zero to One how humans have been able to predict death for over a century, but became complacent in settling for charting it. While LLMs were not around at the time he wrote this book, they still don’t disrupt the longevity field as much as they do math. The AI race is now shortening the verification loop between machine insight and real-world results, because shortening this loop also means compounding it. RSI is all you need.
Interesting. So we need evals/verifiers rooted in the physical world. That makes sense to me.
It seems like agents are always going off the rails because they aren’t given grounded data to anchor their work to. By having them work inside increasingly higher-fidelity representations of the environment they’re meant to be building a solution for, they’ll be better able to self-correct.
Another possible approach to the novelty problem that comes to mind is tuning an agent’s active workspace to match the problem it’s solving. Some problems benefit from broad, lower-fidelity context that preserves global structure, while others benefit from a narrower, higher-fidelity workspace. A hierarchy that can move between those representations could help ideas propagate without relevant contextual details being not taken into consideration.