Pydantic AI 2.64 and 2.65: ModelFallback Falls Back on Refusals, and ArtifactScanner Checks Memory and Skills

Pydantic AI v2.64.0 (published October 30, 2026 per GitHub) added a ModelFallback capability and an ArtifactScanner for agent memory and skills. v2.65.0 (November 3) changed ModelFallback to move on to the next model when a model refuses, not just when its API fails. Two details matter if you run coding or security-adjacent agents: a refusal is now a fallback trigger by default, and the scanner is pattern matching that the docs say misses paraphrased instructions.

I read the v2.64.0 and v2.65.0 release notes, and the Model Fallback, Artifact Scanning, Memory and Skills docs at the v2.65.0 tag, plus the scanner source file. I did not install the package or run anything. These are the docs' claims, not my measurements.

What the release notes list

Both releases put these items under "Compatibility Notes", which means behavior can change on upgrade:

• 2.64.0: a ModelFallback capability with a per-attempt prepare_model_request hook and RetryModelRequest; ArtifactScanner for Memory and Skills; a first_event_timeout model setting so a stream that never starts can fall back; and json-repair becoming an optional dependency.
• 2.65.0: "Fall back to the next model in ModelFallback when a model refuses, and let ContentFilterError in fallback_on match refusals". Also a rule that stored response, item, conversation and container IDs are sent back only to the endpoint that created them (2.64.0), and a Temporal change so model errors that cannot succeed on retry are not retried by default (2.65.0).

ModelFallback, per the docs

ModelFallback is a capability that attempts other models when one fails or returns a response you reject. The chain starts with the model selected for the step, followed by the capability's models. A model already attempted in the step is skipped.

By default it falls back on ModelAPIError and ContentFilterError. fallback_on can also take an exception handler or a response handler that rejects a response that came back successfully. One warning in the docs: passing only a response handler replaces the default, so API errors and refusals are then raised instead of falling back. Pass both in a list to keep them.

Other documented behavior:

• Each model tried is an attempt inside the same request step. usage.requests counts the step once, so UsageLimits.request_limit bounds steps, not models tried. Tokens and cost of every attempt that produced a response, rejected ones included, count in the run's usage.
• The answering response lists the attempts it moved on from in failed_attempts. If every model fails you get a FallbackExceptionGroup.
• By default each step starts again from the first model. sticky=True keeps using the candidate that answered, which the docs tie to keeping the same provider's prompt cache.
• Streaming falls back only before any output reaches you. A streamed response handler is not consulted, and a streamed refusal is accepted as is.

Refusals as a fallback trigger

The docs say Pydantic AI reports a refusal as a successful response with finish_reason of content_filter, with the provider's explanation in provider_details['refusal'] where given. ModelFallback moves on from it by default, to a model that applies its own provider's safety policy. The example in the docs is a coding agent asked to explain a vulnerability in code.

The docs also list limits:

• To have a refusal end the run, pass fallback_on=(ModelAPIError,).
• A plain FallbackModel does not fall back on refusals by default. Pass it fallback_on=(ModelAPIError, ContentFilterError).
• It works per request, with the run's message history so far.
• Without a fallback, an empty refusal raises ContentFilterError, and a refusal with text is the run's output.
• A streamed refusal is not fallen back from. You catch it and rerun with another model=.

The docs link Anthropic's guidance that partial output of a refused response should be treated as incomplete. The design question is yours: whether you want a request refused by one provider silently answered by another. The default now does that. If your policy says a refusal is a stop signal, set fallback_on yourself.

ArtifactScanner

The docs describe it as checking text that an agent writes and a later session loads, such as memory files and skills, for prompt injection, credential exfiltration, and content a reviewer cannot see. Memory and Skills accept it as scanner=. Per the docs, a flagged memory write is refused, a flagged memory file is withheld, and a flagged skill is left out of the catalog.

Detection categories listed: injection, role_override, exfiltration, html_comment and invisible_unicode. The source shows the phrase patterns run after NFKC normalization with invisible and joining characters stripped, so a zero-width character inside a phrase does not hide it. You can narrow categories, add extra_patterns, or subclass and override scan to put another detector behind the same integration.

summary() names threats and pattern names without quoting the artifact, so it can go in a message the model sees. Each flagged decision records a zero-duration artifact_scan flagged span. The matched excerpts and source names go on that span only when trace_include_content is on.

Where the docs say it stops

The Artifact Scanning page is direct about limits:

• "This is pattern matching." Prompt injection is ordinary language, so it misses paraphrased or translated instructions, and an empty result does not mean an artifact is safe.
• It can flag a note that quotes an attack, such as a memory recording that a web page said "ignore previous instructions".
• It scans what the integrations route through it: memory through write_memory, read_memory, search_memory and the injected MEMORY.md, and each skill's description and body. Memory files read with FileSystem and bundled skill files are not scanned unless you call scan.
• For indirect injection in tool results, the page points to a separate Prompt Injection Defender page, which I did not read.

What I would check

These are my untested suggestions, not documented behavior.

• Decide on refusals before upgrading. If you run ModelFallback, write down whether a refusal should fall back, and set fallback_on to match.
• Log failed_attempts and the model request attempt spans. A fallback that works well is invisible unless you look for it.
• Test the scanner with your own memory files, including notes that quote attacks, to see how many false positives you get.
• Treat the scanner as one layer. Keep human review for skills from third parties.
• Check streaming paths separately. Two documented gaps, response handlers and refusals, are both streaming gaps.

What I would not conclude

• I would not conclude the scanner makes memory or skills safe. The docs say it does not.
• I would not conclude other providers refuse less. The docs say the next model applies its own policy.
• I would not conclude anything about the harness package's stability. The docs note the API may change between 0.x minor releases.
• I would not extend any of this to Pydantic AI versions I did not read.

Frequently asked questions

What is ModelFallback in Pydantic AI?

A capability that tries other models when the selected one fails or returns a response you reject. The chain is the step's model followed by the capability's models.

Does it fall back on refusals?

By default yes, as of the 2.65.0 release notes and docs. fallback_on defaults to (ModelAPIError, ContentFilterError). Pass fallback_on=(ModelAPIError,) to opt out.

Does a streamed refusal fall back?

No, per the docs. It is accepted as streamed, and you rerun with another model if you want a fallback.

What does ArtifactScanner check?

Memory and skill text for prompt injection, role override, credential exfiltration patterns, HTML comments and invisible characters.

Can it catch paraphrased injection?

The docs say no. It is pattern matching and misses paraphrased or translated instructions.

Where are the scanner's findings reported?

summary() names threats and patterns without quoting text, and an artifact_scan flagged span records each decision. Excerpts appear on the span only with trace_include_content on.

Did you run it?

No. I read release notes, docs and the scanner source at the tag. I did not install Pydantic AI.

Sources

All read on November 3, 2026.

• pydantic-ai v2.65.0 release notes
• pydantic-ai v2.64.0 release notes
• Model Fallback docs at v2.65.0
• Artifact Scanning docs at v2.65.0
• Memory docs at v2.65.0
• Skills docs at v2.65.0
• ArtifactScanner source at v2.65.0

Not read: the Prompt Injection Defender and Guardrails pages, Anthropic's refusals page, the rest of the 2.64 and 2.65 changes, and anything about running the code.