Google's AQuA Reads Your Production Agent's Traces. What It Does, What It Costs, and What Google Has Not Shown

A production agent can fail without any error. It returns HTTP 200, stays inside its latency budget, and still records a seat as confirmed without checking whether the seat is free.

On October 8, Google published AQuA (Ambient Quality Agent), an open-source reference implementation that tries to catch that class of failure. Everything below comes from Google's post and the repository it points to. The results are Google's own, from its own demo agent.

What AQuA is

According to the post, AQuA runs unattended beside your agent in your Google Cloud project. It reads production trajectories from Cloud Trace, Cloud Logging or BigQuery on a schedule, after a deployment, or on demand. Google says it never sits in the request path and never writes back to your agent.

Each run goes through five stages:

1. Sample. Up to 1,000 recent sessions, picked at random, to keep cost predictable.

2. Review. Each session is graded against a nine-point checklist (procedure, tool selection and arguments, grounding, task completion and others). A failure becomes a structured actual-versus-expected finding.

3. Cluster. Findings with the same failure mechanism are grouped.

4. Verify. A separate model checks each cluster against up to three full transcripts and drops clusters the evidence does not support.

5. Track. Surviving clusters are matched against open insights in BigQuery as new, recurring, or resolved after 14 days unseen.

You can steer the review with a plain-English goal.md, and add deterministic Python metrics in eval_config.yaml. Root-cause analysis is a separate on-demand step. It reads failing trajectories next to an immutable source snapshot captured at deploy time, and when the defect is in your repo it cites file and line ranges and proposes an edit. Google says it never applies an edit or opens a pull request on its own.

Google's worked example

The post walks through travel-concierge, a multi-agent sample in google/adk-recipes. Google replayed four scripted traveler journeys 32 times, which produced 1,583 OpenTelemetry spans.

• 5 sessions passed and 27 produced 42 findings.
• Clustering gave 9 candidate clusters. The verifier rejected 3 and kept 6.
• The top issue: the planning agent saved a seat number to session state without calling the seat-selection agent (15 sessions).
• Another: a vegan food preference in the traveler profile never reached a sub-agent wrapped as an isolated tool, so it suggested steak dishes (7 sessions).
• A third: a sub-agent was declared with an empty tool list while its prompt required verified map fields, so it made up placeholder example.com URLs (5 sessions).

Google says root-cause analysis traced the first two to single lines in prompt files. After two one-line prompt fixes and a redeploy, replaying the same 32 sessions showed the seat issue dropping from 15 sessions to 2 and the vegan issue from 7 to 0. Full-session passes went from 5 of 32 to 13 of 32.

Note the scale. This is 32 scripted replays on a sample agent, not a measurement on real traffic. The post itself calls the system a reference implementation with deliberate trade-offs.

Cost, as Google reports it

Google gives two run costs at standard Gemini platform pricing:

• A 96-session single-agent sweep: $0.70 total, about $0.007 per session.
• The 32-session travel-concierge sweep: $3.76 total, about $0.12 per session.
• Root-cause diagnosis is on demand and costs $0.33 to $2.47 per investigated insight, depending on how many full trajectories it reads.

The per-session cost rose about 17x between the two examples, and Google ties that to trajectory depth (about 5 to 6 spans per session versus about 50). Your agents will land somewhere on that curve. I have not run AQuA, so I cannot say what your number would be.

What Google says it has not solved

The post lists its own open problems. The ones that matter for deciding whether to adopt it:

• Random sampling. It samples up to 1,000 sessions at random. Google says catching a silent regression that affects 1% of a critical workflow, without running a deep judge on every session, is still hard.
• Judges versus experts. Getting model judges to agree with domain experts is real engineering work.
• Replay limits. adk run --replay resends recorded user turns, which works when tools are idempotent or mocked, but does not rebuild external state.
• Long traces. Passing whole transcripts breaks down for agents that run hundreds of turns.
• Verifier error. On an 87-trace internal benchmark, the verifier rejected 4 of 24 candidate clusters. Google says an agent that diagnoses your agent will sometimes be wrong.

What I would check before adopting it

These are my own untested suggestions.

1. Read your own failures first. If you do not already know your top three failure patterns by hand, a tool that clusters them will be hard to evaluate.

2. Check the fit. AQuA is scaffolded one to one beside an ADK agent on Google Cloud. If your agent is on another stack, Google's post does not say how much work the port is.

3. Run the local demo. Google says make demo serves the dashboard on synthetic data with no cloud project, credentials or model calls.

4. Treat insights as leads. Open a few cited traces yourself before acting on a root cause.

What I would not conclude

• That the pass-rate jump from 5/32 to 13/32 predicts anything on real traffic.
• That the cost figures hold for your agent.
• That an automated diagnosis replaces reading traces. Google's own framing is a first pass for whoever is on rotation.

Frequently asked questions

What is AQuA?

An open-source ambient quality agent from Google that sweeps production agent sessions, clusters failures, verifies them and can diagnose root causes against deploy-time source.

Where does it run?

In your Google Cloud project, beside your agent. Google says transcripts, source snapshots and BigQuery tables stay inside your project.

Does it change my agent?

Per Google, no. It is outside the request path and never applies edits or opens pull requests itself.

Where does it get its data?

Cloud Trace, Cloud Logging or BigQuery.

How much does it cost to run?

Google reports $0.007 to $0.12 per session in its two examples, plus $0.33 to $2.47 per root-cause investigation.

Can I try it without a cloud project?

Google says the local demo on synthetic data needs no cloud project, credentials or model calls.

Is it production-ready?

Google describes it as a reference implementation shared in the open to collect feedback, with several listed open problems.

Sources

• Google Developers Blog, The Outer Loop, Insights First: An Ambient Quality Agent That Diagnoses Your Production Agent (October 8, 2026): all claims and numbers above.
• google/adk-recipes, ambient-quality-agent: the repository the post points to. I did not review its contents beyond the link.