The Smartest Model Is Not What Makes AI Agents Reliable

By Harel Asaf · Published June 10, 2026 · Updated June 10, 2026

What makes AI agents reliable is not the smartest model. It is the deterministic layer underneath: the software that fetches, filters, and verifies the same way on every run. Two Anthropic papers, released in the same week, show a cheaper model paired with the right deterministic tool beat an expensive model alone, and gave the same answer every time. That is the whole game.

TL;DR

• A general model matched decades-old chemistry software, but frontier agents failed a biology retrieval task where the passing bar is 100%.
• The fix was not a bigger model. It was a deterministic retrieval tool that ran the same way every time.
• With that layer in place, every agent cleared 90%, the top result hit 99.7%, and run-to-run variance mostly disappeared.
• A cheaper model plus the right tool matched the expensive model, improving reliability and cost in the same move.
• Split every workflow into steps that need creativity and steps that must be exact. Hand the exact steps to a deterministic tool.

A cheaper model with the right deterministic tool beat the expensive model alone, and it gave the same answer every time. That is the whole game.

In one week, Anthropic put Claude to work as a chemist and as a biologist. Same lab, two papers, two very different results. As a chemist, a general model matched software that took decades to build. As a biologist, the same class of model failed a task where the passing grade is 100%, and failed it differently every time it tried.

The gap between those two outcomes is the most useful thing I have read about deploying AI agents this year.

What did the two papers actually find?

According to Anthropic's "Making Claude a chemist," a general-purpose model now does routine chemistry as well as the specialized tools chemists have relied on for years. The team tested Claude against ChemDraw and MestReNova on predicting NMR spectra for 20 compounds pulled from preprints published after the model's training cutoff. On hydrogen shifts, Opus 4.7 landed within ±0.079 ppm on average, well under half the window a chemist would call correct. It matched the experimental splitting pattern more often than either dedicated tool. Then it did the harder thing the software does not do at all: it worked backward from a spectrum to the structure, recovering all eight of the simpler target molecules on every single attempt.

The paper is careful about what this means.

Claude is starting to meaningfully assist chemists with the daily translation, recall, and integration work that complements their judgment.

That is a precise claim. Not "AI does chemistry now." It does the translation, recall, and integration work. The judgment stays with the chemist, and the model shows its reasoning step by step so the chemist can check it.

Now the other paper. According to Anthropic's "Paving the way for agents in biology," the same generation of agents fell apart on what sounds like a simpler task: pulling the right virus sequences out of a public database. The team built a benchmark called VirBench, 120 realistic queries across 40 pathogens, and let frontier agents loose on it. Accuracy ranged from 16.9% to 91.3%. For this kind of work the real bar is 100%, because a single missing record can move the inferred start of an outbreak by weeks. Worse than the average was the variance. Asked the same Ebola query three times, one model returned 266 sequences, then 15, then 5. One bad pull pushed the inferred root of the 2014 outbreak back to the year 1922.

Here is the line that matters:

The bottleneck for biological agents is not only reasoning but the absence of widespread deterministic execution layers for querying biological data.

So the team built one. A retrieval tool called gget virus that handles the messy database logic deterministically. With that layer in place, every agent cleared 90%, the top result hit 99.7%, and the run-to-run variance mostly disappeared. The gap between models nearly closed.

What does this mean for teams building AI agents today?

Read those two results next to each other and a pattern shows up that has nothing to do with chemistry or biology.

Where the model could reason its way through and show its work, it won. Where success depended on navigating brittle, undocumented, human-shaped infrastructure, raw intelligence was not the fix. A deterministic layer was. And once that layer existed, the expensive model stopped mattering. A cheaper one paired with the right tool produced the same answer, reliably, every time.

This is the part most teams get backward. The instinct when an AI workflow is flaky is to reach for a bigger model. The biology paper shows the bigger model is often not the lever. The lever is the boring layer underneath: the part that fetches, filters, and verifies the same way on Tuesday as it did on Monday. You can build that layer in n8n, in a Make scenario, in a few hundred lines of Python wrapped around your real data source, or in a retrieval tool your agent calls instead of improvising. The point is that the part that has to be correct should not be left to a model to reinvent on every run.

There is a cost argument hiding in here too, and it is a strong one. If a deterministic tool lets a cheaper model match an expensive one, your reliability and your bill both improve at the same time. Model routing stops being the interesting decision. Tool design becomes the interesting decision.

What does a deterministic layer still not fix?

A model that can muscle through a confusing workflow is not the same as a workflow you can trust. The biology paper says this directly:

A model that can fight its way through a confusing bioinformatics workflow may still be too expensive, too slow, too hard to audit, or too difficult to trust for routine scientific work.

Auditability is the word I keep circling. The chemistry result was usable not only because it was accurate but because the model showed its reasoning, peak by peak, so an expert could check it. The biology failures were dangerous precisely because the wrong answers looked plausible. An agent that confidently returns 15 sequences when the truth is 266 has not given you an error. It has given you a clean-looking result that is silently wrong, sitting at the top of a much longer pipeline.

That is the design problem. Not "is the model smart enough," but "when it is wrong, will anyone be able to tell." Reproducibility and a visible trail are not nice-to-haves you add at the end. They are the difference between a demo and a system you can put in front of a regulator, an auditor, or a customer.

The practitioner take

I have spent two years building these workflows by hand, and the thing I keep relearning is that the model is the easy part. The reliability lives in everything around it.

These two papers are the cleanest proof of that I have seen, because they come from the company that makes the model, and the lesson cuts against its own product. One paper shows the model is genuinely good now. The other shows that when the model is not enough, the answer is not a smarter model. It is a deterministic tool, version-controlled, documented, and built to be called the same way every time.

The thread connecting both is a single idea: creativity and reliability are different jobs, and you should not ask the same component to do both. Let the model generate hypotheses, read the figure, propose the structure, draft the answer. Hand the parts that must be exact to a layer that cannot drift. The biology team put it better than I can, that the layer underneath the creativity has to be boringly reliable. Boring is the feature.

For anyone deploying agents inside a real organization, this reframes the build. The question is not which model to standardize on. It is: which steps in this workflow are creative, and which steps must be correct every single time? Draw that line, and most of your reliability problem is already solved before you pick a model at all.

Key terms explained

Term	What it means in plain language
Deterministic layer	A piece of software that does the same thing every time it runs, with no variation, unlike a model that can answer differently on repeat runs.
NMR spectrum	A standard chemistry readout used to figure out a molecule's structure. Reading it is slow, manual expert work.
Structure elucidation	Working backward from a spectrum to the molecule that produced it. The hard direction, which most software leaves to the human.
Reproducibility	Getting the same answer when you ask the same question twice. The thing the biology agents lacked, and the thing the deterministic tool restored.

What to do with this

Start by mapping any AI workflow you run into two columns: the steps that need creativity and the steps that need to be exactly right. The split is usually obvious once you force it, and it tells you where a model belongs and where a deterministic tool belongs. Then look hard at your flaky workflows before you reach for a bigger model, because the biology result suggests the model is often not the constraint and a tool around your real data source will move reliability further than an upgrade will. While you are there, treat auditability as a requirement and not a bonus, since a wrong answer that looks right is more dangerous than an obvious failure, and a visible reasoning trail is what let the chemistry result be trusted in the first place. Finally, run the cost math, because if a cheaper model plus the right tool matches an expensive model alone, you have just bought reliability and a smaller bill in the same move.

The bigger picture

We spent two years watching the models get smarter and assuming that was the story. These two papers, shipped in the same week by the people building the models, quietly tell a different one. The frontier that matters for real work is not the next model. It is the unglamorous infrastructure that turns a capable model into a system you can trust on Monday morning with messy inputs and someone's job on the line.

The smartest thing in your stack should be allowed to be creative. Everything that has to be correct should be too boring to surprise you.

Frequently asked questions

What makes AI agents reliable?

Reliability comes from the deterministic layer around the model, not the model itself. A deterministic tool fetches, filters, and verifies data the same way on every run. In Anthropic's biology work, adding that layer pushed every agent past 90% accuracy and removed most of the run-to-run variance.

Does a bigger model fix a flaky AI workflow?

Usually not. The biology paper showed frontier models failing a retrieval task regardless of size, because the bottleneck was brittle infrastructure, not reasoning. A deterministic retrieval tool fixed it. Reach for tool design before a model upgrade when a workflow is unreliable.

What is a deterministic layer in an AI agent?

It is software that does the same thing every time it runs, with no variation. It handles the steps that must be exact, like querying a database or validating a record, so the model is not left to reinvent that logic on each run. Think n8n, a Make scenario, or a wrapped Python function.

Why did a cheaper model match an expensive one?

Once a deterministic tool handled the hard, exact part of the task, the model only had to do the parts it was already good at. With the variance removed by the tool, the expensive model's edge disappeared, so a cheaper model produced the same reliable answer at lower cost.

What did "Making Claude a chemist" show?

Claude matched dedicated tools like ChemDraw and MestReNova at predicting NMR spectra, landing within ±0.079 ppm on hydrogen shifts. It also worked backward from a spectrum to the structure, recovering all eight simpler target molecules on every attempt, while showing its reasoning so a chemist could check it.

What did "Paving the way for agents in biology" show?

On VirBench, 120 queries across 40 pathogens, frontier agents scored between 16.9% and 91.3% with high variance. One Ebola query returned 266, then 15, then 5 sequences across three runs. A deterministic retrieval tool called gget virus lifted every agent past 90%.

Why does auditability matter for AI agents?

A wrong answer that looks right is more dangerous than an obvious failure. The chemistry result was trusted because the model showed its reasoning peak by peak. The biology failures were risky because plausible but wrong results sat silently at the top of a longer pipeline.

How do I decide which steps a model should handle?

Map the workflow into two columns: steps that need creativity and steps that must be exact every time. Give the creative steps to the model, such as drafting or hypothesis generation. Give the exact steps to a deterministic tool. The split is usually obvious once you force it.

Is this only relevant to scientific AI work?

No. The pattern is domain-agnostic. Any enterprise AI workflow that touches brittle data sources, compliance steps, or anything that must be correct every run benefits from a deterministic layer. The science papers are just an unusually clean proof of a general deployment principle.

Where should teams invest first for agent reliability?

Invest in the deterministic tooling around your real data source before standardizing on a model. Version-control it, document it, and make it callable the same way every time. That layer moves reliability further than a model upgrade, and it usually lowers cost at the same time.

Sources and credits

All research referenced is the intellectual property of the respective organizations. This article represents Harel Asaf's independent practitioner analysis. The two Anthropic science papers are featured because they form a single, coherent thread for teams deploying AI agents; the DeepMind and NVIDIA publications below were also detected the same week and are credited for completeness.

Lab	Publication title	URL	Date
Anthropic	Paving the way for agents in biology	https://www.anthropic.com/research/agents-in-biology	Jun 8, 2026
Anthropic	Making Claude a chemist	https://www.anthropic.com/research/making-claude-a-chemist	Jun 5, 2026
Google DeepMind	Solipsistic superintelligence is unlikely to be cooperative	https://deepmind.google/research/publications/231466	Jun 4, 2026
NVIDIA Research	NVIDIA Research Unlocks Advanced Grasping, Smarter Autonomous Driving and Agent Training at Scale	https://blogs.nvidia.com/blog/cvpr-research-grasping-driving-agent-training/	Jun 3, 2026

---

Harel Asaf is an AI systems builder with a legal background. He focuses on operational AI, the systems that survive Monday morning. Based in Tel Aviv.

type="application/ld+json"

dangerouslySetInnerHTML={{

__html: `{

"@context": "https://schema.org",

"@type": "Article",

"headline": "The Smartest Model Is Not What Makes AI Agents Reliable",

"description": "AI agents reliability comes from a deterministic layer, not a bigger model. Two Anthropic papers show a cheaper model plus the right tool wins every run.",

"author": { "@type": "Person", "name": "Harel Asaf", "url": "https://harelasaf.com/about" },

"datePublished": "2026-06-10",

"dateModified": "2026-06-10",

"mainEntityOfPage": { "@type": "WebPage", "@id": "https://harelasaf.com/articles/ai-agents-reliability-deterministic-layer" },

"publisher": { "@type": "Person", "name": "Harel Asaf" },

"keywords": "AI agents reliability, deterministic layer, enterprise AI workflow, AI agent governance"

}`,

}}

/>

type="application/ld+json"

dangerouslySetInnerHTML={{

__html: `{

"@context": "https://schema.org",

"@type": "FAQPage",

"mainEntity": [

{ "@type": "Question", "name": "What makes AI agents reliable?", "acceptedAnswer": { "@type": "Answer", "text": "Reliability comes from the deterministic layer around the model, not the model itself. A deterministic tool fetches, filters, and verifies data the same way on every run. In Anthropic's biology work, adding that layer pushed every agent past 90% accuracy and removed most of the run-to-run variance." } },

{ "@type": "Question", "name": "Does a bigger model fix a flaky AI workflow?", "acceptedAnswer": { "@type": "Answer", "text": "Usually not. The biology paper showed frontier models failing a retrieval task regardless of size, because the bottleneck was brittle infrastructure, not reasoning. A deterministic retrieval tool fixed it. Reach for tool design before a model upgrade when a workflow is unreliable." } },

{ "@type": "Question", "name": "What is a deterministic layer in an AI agent?", "acceptedAnswer": { "@type": "Answer", "text": "It is software that does the same thing every time it runs, with no variation. It handles the steps that must be exact, like querying a database or validating a record, so the model is not left to reinvent that logic on each run. Think n8n, a Make scenario, or a wrapped Python function." } },

{ "@type": "Question", "name": "Why did a cheaper model match an expensive one?", "acceptedAnswer": { "@type": "Answer", "text": "Once a deterministic tool handled the hard, exact part of the task, the model only had to do the parts it was already good at. With the variance removed by the tool, the expensive model's edge disappeared, so a cheaper model produced the same reliable answer at lower cost." } },

{ "@type": "Question", "name": "What did Making Claude a chemist show?", "acceptedAnswer": { "@type": "Answer", "text": "Claude matched dedicated tools like ChemDraw and MestReNova at predicting NMR spectra, landing within 0.079 ppm on hydrogen shifts. It also worked backward from a spectrum to the structure, recovering all eight simpler target molecules on every attempt, while showing its reasoning so a chemist could check it." } },

{ "@type": "Question", "name": "What did Paving the way for agents in biology show?", "acceptedAnswer": { "@type": "Answer", "text": "On VirBench, 120 queries across 40 pathogens, frontier agents scored between 16.9% and 91.3% with high variance. One Ebola query returned 266, then 15, then 5 sequences across three runs. A deterministic retrieval tool called gget virus lifted every agent past 90%." } },

{ "@type": "Question", "name": "Why does auditability matter for AI agents?", "acceptedAnswer": { "@type": "Answer", "text": "A wrong answer that looks right is more dangerous than an obvious failure. The chemistry result was trusted because the model showed its reasoning peak by peak. The biology failures were risky because plausible but wrong results sat silently at the top of a longer pipeline." } },

{ "@type": "Question", "name": "How do I decide which steps a model should handle?", "acceptedAnswer": { "@type": "Answer", "text": "Map the workflow into two columns: steps that need creativity and steps that must be exact every time. Give the creative steps to the model, such as drafting or hypothesis generation. Give the exact steps to a deterministic tool. The split is usually obvious once you force it." } },

{ "@type": "Question", "name": "Is this only relevant to scientific AI work?", "acceptedAnswer": { "@type": "Answer", "text": "No. The pattern is domain-agnostic. Any enterprise AI workflow that touches brittle data sources, compliance steps, or anything that must be correct every run benefits from a deterministic layer. The science papers are just an unusually clean proof of a general deployment principle." } },

{ "@type": "Question", "name": "Where should teams invest first for agent reliability?", "acceptedAnswer": { "@type": "Answer", "text": "Invest in the deterministic tooling around your real data source before standardizing on a model. Version-control it, document it, and make it callable the same way every time. That layer moves reliability further than a model upgrade, and it usually lowers cost at the same time." } }

]

}`,

}}

/>