Agent Orchestration Patterns That Actually Work in Production

Most of what's written about agent orchestration is written by people who haven't run agents in production. I've been running a 7-agent autonomous system since early 2026 — daily article publishing, LinkedIn content, financial monitoring, web design, sales ops, infrastructure. These three patterns are the ones that survived contact with reality.

They're not the most elegant patterns I've tried. They're the ones that didn't break at 6 AM when nobody was watching.

---

Why Orchestration Is the Hard Part Nobody Talks About

Building a single agent is, honestly, not that hard anymore. You pick a model, write a system prompt, hook it up to tools. Done in an afternoon.

Orchestration is the hard part. The moment you have two or more agents that need to hand off work, share memory, or coordinate on timing — you have an orchestration problem. And the solutions that look clean on a whiteboard fall apart fast.

I spent about three weeks in early 2026 trying to get agents to communicate directly — agent A calls agent B, B returns a result, A continues. It worked in demos. It failed in production because of latency, because of partial failures, because when B timed out, A had no recovery path. The system got into states I couldn't debug.

So I scrapped that. What I run now is simpler, more boring, and dramatically more reliable.

---

Pattern 1 — The Task Backlog + Router Agent

This is the foundation of everything else. Instead of agents calling each other directly, all cross-agent work flows through a shared backlog. When one agent needs another to do something, it writes a task to the backlog. A router agent — Vision, in my system — reads the backlog periodically and assigns tasks to the right agent.

Why this works:

• Agents are decoupled. Aria doesn't call Jams. Aria writes "create LinkedIn post for this article" to the backlog, and Vision routes it when Jams is available.
• The backlog is auditable. Every task has a timestamp, a status, an assignee. When something breaks, I can see exactly what happened.
• Recovery is trivial. If an agent fails mid-task, the task sits in the backlog as PENDING and gets picked up on the next cycle.

Real example: When Aria publishes an article (this one, right now), she creates a backlog task for Jams to write a LinkedIn post announcing it. Jams processes that task in her next scheduled run. The two agents never talk directly. The whole handoff is a structured message in a shared store.

The backlog itself lives in Firestore. Each task document has: to (agent name), task (plain English description), priority (urgent/high/normal/low), status (pending/done), createdAt, completedAt. That's it. No complex schemas.

Numbers: In a typical week, Vision processes 40–60 backlog items. Median time from task creation to completion: 14 minutes during active hours, under 2 hours overnight. Task failure rate: about 3%, almost all recoverable on retry.

---

Pattern 2 — Scheduled Autonomous Runs

Direct calls between agents fail silently. Scheduled runs don't.

Each of my agents has a defined schedule. Aria runs her content pipeline every weekday at 06:30 IL. Albert runs a financial sweep every morning. The llm-citation-tracker runs weekly. These aren't webhooks. They're Cloud Scheduler jobs that hit dedicated endpoints on Cloud Run services.

This pattern solved a problem I didn't know I had until I hit it: agent drift. When Agent A triggers Agent B by calling it, the timing is unpredictable. If A runs slow one day, B gets called late. If A crashes, B never runs. Over a week, the whole system gets out of sync.

Scheduled runs eliminate timing dependencies. Each agent is responsible for its own schedule. The backlog handles coordination. Schedules handle cadence.

The actual setup:

• Cloud Scheduler triggers a POST to /cron/aria-daily-content every weekday at 06:30 +03:00
• Cloud Run service receives the request and boots the agent context
• Agent reads its backlog, reads memory, does its work, writes outputs
• Cold start is 1.8–2.4 seconds (acceptable for cron jobs that aren't user-facing)
• Total end-to-end for Aria's daily content run: 3–7 minutes depending on article complexity

Scratch that — my original setup had a single Cloud Scheduler job that tried to coordinate all agents in sequence. It was a disaster. One slow agent would time out the whole chain. Now each agent has its own scheduler job, its own Cloud Run service, its own lifecycle. Complete independence.

Numbers: 7 agents, 14 scheduled jobs total (some agents run twice daily). Combined Cloud Run cost for scheduled runs: $12–$22/month depending on the month. One month I left a debug endpoint running and saw the bill hit $31 — caught it with LLM Cost Lens and shut it down within the hour.

---

Pattern 3 — Cross-Agent Memory via Firestore

This is the one I'm most proud of, and the one that has the most interesting failure modes.

Agents share memory through Firestore. There are three types of memory in my system: learning (insights discovered), decision (strategic choices made), feedback (audience reactions). Any agent can read from or write to any memory collection.

Here's why this matters for orchestration: agents that ran weeks ago inform agents running today. When Jams writes a learning that says "posts about agent failure states get 3× more engagement than technical how-to posts," Aria reads that before writing articles. When Albert flags a budget overrun, Vision reads it before scheduling expensive API-heavy tasks.

Memory is how the system gets smarter over time without me manually briefing every agent. It's also how agents stay aligned without direct communication.

The pattern in practice:

1. Every agent reads relevant memory at the start of each run (decisions, learnings, feedback)

2. Every agent writes memory at the end of each run if it discovered something worth keeping

3. Memory entries are typed, timestamped, and written in natural language — easy for LLMs to parse

4. No memory entry is ever deleted — they accumulate, and recency bias in retrieval keeps them relevant

Where it breaks: Memory pollution. If agents write low-quality insights, the memory collection fills up with noise. I had a period in June where Albert was writing a learning entry on every run ("cost this week was $X") — useful data, but not a learning. I added a quality gate: before writing to memory, agents are instructed to ask "would this still matter in 4 weeks?" If no, skip.

Numbers: Current memory store has 287 entries across 3 types. Typical agent reads 8–12 entries per run (the most recent of each type). Firestore cost for memory operations: $0.80–$1.20/month. Absurdly cheap for what it provides.

---

The Pattern That Didn't Make the List

I tried one more pattern that I thought would be clever: hierarchical task decomposition. Agent receives a big task, breaks it into subtasks, spawns sub-agents to handle each one, aggregates results.

It works in theory. In practice, the latency compounds. If a task breaks into 4 subtasks, and each subtask takes 30 seconds, you're looking at 2 minutes of parallel work plus coordination overhead. Fine for offline batch work, bad for anything time-sensitive. And the error handling is genuinely painful — if subtask 3 fails, what do you do with the results from 1, 2, and 4?

I still use decomposition occasionally, but I do it manually — I break tasks up front and write separate backlog items. Less automatic, more reliable.

---

How These Three Patterns Connect

It's not three independent patterns. It's one system:

Backlog + router = how agents coordinate work

Scheduled runs = how agents coordinate time

Shared memory = how agents coordinate knowledge

Remove any one of them and the system gets dumber. Remove the backlog and you're back to agents calling each other directly — fragile. Remove schedules and work happens unpredictably. Remove shared memory and every agent starts fresh every run, rediscovering things the system already knows.

In practice, a typical day looks like this: Aria's 06:30 run fires. She reads memory (recent decisions, feedback on past articles). She picks a topic from the queue, checks the dedup gate, writes the article, commits it. She creates a backlog task for Jams to write a LinkedIn post. She saves a learning if she discovered something worth keeping. End of run.

Four hours later, Jams's run fires. She reads the backlog, finds Aria's task, writes the LinkedIn post, marks the task done. She reads memory to check engagement patterns before writing. She saves feedback after publishing.

No direct calls. No shared state except the backlog and memory. Each agent runs in its own process, on its own schedule, with a clear interface to the shared system.

---

What I'd Tell Someone Starting From Scratch

Don't start with the patterns. Start with one agent doing one thing well.

I ran a single Cloud Run service handling WhatsApp messages for three months before I added a second agent. That constraint forced me to understand the infra, the failure modes, the latency characteristics. When I added agents 2 through 7, I was building on something I actually understood.

The patterns above emerged from real problems. The backlog came from agents calling each other and failing. Schedules came from timing drift. Memory came from agents forgetting context between runs. If I'd read a blog post and tried to implement all three from day one, I'd have built something I couldn't debug.

Build one agent. Break it. Fix it. Add the next one.

I was wrong about this for a while — I thought the architecture was the first thing to get right. It's the last. The first thing to get right is understanding what failure looks like for your specific system. Architecture follows.

---

ctxauditor's Role in All of This

One thing I haven't mentioned: context management is the hidden fourth layer of orchestration.

Each of these agents runs with a context window. As they read memory, read backlogs, read tool outputs, that context fills up. Without management, agents hit their limits mid-run and start hallucinating or refusing to complete tasks.

ctxauditor — one of my prototypes — runs a relevance check on agent context before each major operation. It reduced irrelevant context from 38% to about 12% across my agents. That's the difference between an agent that finishes its run cleanly and one that runs out of context window 70% of the way through and produces garbage.

The three patterns above assume context management is handled. If you're implementing these patterns and skipping that step, you'll hit walls I already hit.

---

Sources

These references support the technical concepts discussed here. They do not independently verify the personal experiments, costs or performance figures reported in this essay. Product behavior can change after publication.

• Building effective agents: orchestration patterns and tradeoffs.
• Scheduled services: scheduled invocation on Cloud Run.

FAQ

What is agent orchestration?

Agent orchestration is how you coordinate multiple AI agents working on related tasks. Instead of one AI doing everything, you have specialized agents — each with a defined role — that hand off work, share memory, and operate on schedules. Orchestration defines how they communicate without stepping on each other.

What's the difference between agent orchestration and multi-agent systems?

Multi-agent system describes what you have — multiple agents. Orchestration describes how they work together — the patterns, protocols, and infrastructure that coordinate them. You can have a multi-agent system with bad orchestration. The goal is the former with intentional design for the latter.

Why use a task backlog instead of direct agent-to-agent calls?

Direct calls create tight coupling: if Agent B is slow or fails, Agent A is blocked. A shared backlog decouples them. Agent A writes a task and moves on. Agent B picks it up on its next scheduled run. The system continues operating even if one agent is down, and every hand-off is auditable.

How do I prevent agents from writing bad data to shared memory?

Add a quality gate to the memory-write instruction: before writing a learning or decision, the agent should ask "would this still be relevant in 4 weeks?" If no, skip the write. Also, periodically audit memory entries — I do this monthly — and prune ones that were useful at the time but are now outdated.

What infrastructure do I need to run scheduled AI agents?

My setup: Cloud Run (agent services), Cloud Scheduler (cron triggers), Firestore (backlog + memory). Total infra cost: $12–$31/month depending on usage. You could use AWS Lambda + EventBridge + DynamoDB for a similar setup. The pattern matters more than the specific tools.

How do I handle agent failures in a scheduled orchestration system?

Design for idempotency. Every task in the backlog has a status field. If an agent crashes mid-run, the task stays PENDING and gets picked up on the next cycle. Don't delete tasks on failure — mark them FAILED with an error note. Most failures are transient and recover on retry within 1–2 cycles.

Can I start with just one of these three patterns?

Yes. Start with the task backlog. It's the highest-leverage pattern because it decouples your agents immediately. Add scheduled runs once you have 2+ agents with distinct cadences. Add shared memory once you want agents to learn from each other across sessions. Each pattern is independently useful.

How does context window management fit into agent orchestration?

Context is the hidden constraint in every orchestration pattern. Agents reading from backlogs, memory, and tool outputs fill their context window fast. Without active context management — pruning irrelevant entries before they accumulate — agents degrade mid-run. ctxauditor reduced irrelevant context in my system from 38% to 12%, which directly improved run completion rates.