Agent Orchestration Patterns

01Three orchestration patterns

Hierarchical

A supervising agent decomposes a task and delegates to specialist sub-agents; the supervisor reconciles their outputs. Best for goals with clear decomposition (multi- step research, incident triage, code review). Predictable. Auditable. Slower than the alternatives at small task sizes.

Swarm

A pool of peer agents claims and processes work items from a shared queue. No supervisor; coordination is implicit via shared state. Best for embarrassingly parallel tasks (large-corpus enrichment, fleet-wide patching, bulk alert triage). High throughput. Hard to debug. Easy to overload.

Pipeline

Agents are arranged as a directed graph; each stage processes input and passes output to the next. Best for deterministic transformations (ingest → enrich → classify → act). Low variance. Easy to test. Brittle when stages need to disagree.

02Workload fitness — when to use which

We sorted 4,200 production agent deployments into a fitness matrix. The pattern below explains what worked, not what any single team chose first.

Use Hierarchical when…

• The task has 3–7 distinct sub-goals
• Outcomes need to be reconciled before acting
• The audit trail needs to read like a memo, not a log
• Latency budget ≥ 5s end-to-end

Use Swarm when…

• Items are independent and abundant (> 100/min)
• Per-item cost matters more than per-item quality
• Failures are tolerable individually
• You have a strong rate limiter and a poison-pill detector

Use Pipeline when…

• The transformation is well-defined and stable
• Each stage is testable in isolation
• Latency variance is the dominant cost
• Compliance requires stage-level evidence

03Benchmarks across 4,200 deployments

Numbers are medians across the cohort, instrumented at the orchestrator level. Single-agent calls are excluded.

Throughput (tasks per agent-hour)

• Hierarchical — 18 / agent-hour
• Swarm — 240 / agent-hour
• Pipeline — 110 / agent-hour

Median end-to-end latency

• Hierarchical — 11.4s
• Swarm — 1.8s
• Pipeline — 3.6s

Cost per task (normalized to a baseline LLM call)

• Hierarchical — 4.7×
• Swarm — 1.1×
• Pipeline — 2.3×

Decision-quality (analyst-rated, 5-point scale)

• Hierarchical — 4.4
• Swarm — 3.2
• Pipeline — 3.9

The spread is honest: hierarchical pays 4.3× the cost for a 1.2-point quality gain. Whether that's worth it depends entirely on whether the workload tolerates 3.2 or needs 4.4.

04The hybrid pattern that won most often

In practice, the deployments that performed best across the cost/quality/latency triangle did not pick one pattern. They composed two:

• Pipeline → Hierarchical. Pipeline handles ingestion and classification cheaply; only the items that pass a threshold get escalated to a hierarchical orchestrator that can reason. This is the SOC pattern: 90% of alerts close in pipeline; 10% invoke the supervisor.
• Swarm → Pipeline. Swarm enriches a large corpus in parallel; results flow into a pipeline that normalizes, dedupes, and persists. This is the threat-intel pattern: ingest fans out, dedup converges.

MAESTRO supports composition explicitly: a hierarchical supervisor can spawn a swarm; a pipeline stage can be a pipeline. The cost of the abstraction is two days of learning; the payoff is choosing the right tool for the right segment of the workload.

05Four anti-patterns

• The infinite hierarchy. A supervisor that delegates to sub-supervisors that delegate to sub-sub-supervisors. Each level adds latency and a failure mode. Three levels is almost always too many.
• The unbounded swarm. A swarm with no rate limiter, no poison-pill detection, and no quota. The first prompt-injection in the queue blooms into a fleet-wide outage in minutes.
• The pipeline that should have been one prompt. Five stages, each calling an LLM, doing what a single well-prompted call would do at 1/5 the cost. Pipelines pay overhead at every hop; the right number of stages is “as few as possible”.
• The agent-as-microservice.Wrapping a deterministic function in agent chrome because “everything should be an agent.” If the function would work as code, it should be code. Save the agent budget for tasks that need reasoning.

06Observability as a first-class concern

Every orchestration pattern fails differently, and the observability needs follow.

• Hierarchical— record the supervisor's decomposition explicitly. The most common debug starts with “why did the supervisor break it down that way?”
• Swarm — track per-task throughput and queue age. The failure mode is throughput collapse, not a single broken task.
• Pipeline — measure stage variance, not just totals. A pipeline whose median is 3.6s and whose p99 is 47s is a pipeline with a sleeping stage.

All three need: per-action attestation, per-LLM-call cost attribution, and a flatten-to-trace view that lets a human read the run as a story.

07Closing

The right pattern depends on the workload. The right fabric is the one that lets you change patterns when the workload changes — and shows you why one outperformed the other while you were debating it. That fabric is what MAESTRO is.