Fan-Out Geometry: Where Parallel Pipelines Fail in 2026

Seventy percent of production multi-agent deployments lean on a central orchestrator, according to aiagentsnews.top. That orchestration bias is a warning: pure parallel fan-out is beautiful in demos and brittle in probability. At a 10% per-branch failure rate, a maximum-width split creates a real chance that at least one branch dies, turning a genuine speedup into a self-inflicted error tax.

The February wave of multi-agent launches made parallel execution the default. But a Google/MIT study showed that independent multi-agent systems amplify errors compared with the single-agent baseline, and capability saturates around 45% single-agent accuracy. Past that threshold, extra agents bring diminishing or negative returns; the saturation point is the cliff.

The practical response is not maximum fan-out; it is the narrowest fan-out that still captures the latency win. Staying below the saturation cliff and keeping orchestration in charge lets teams collect most of the parallel advantage at a fraction of the failure risk. The maximum-width era is real, but the default should be deliberately narrower.

Fan-Out Geometry

The geometry of a fan-out decides where a parallel pipeline fails, and in Temporal 1.24's parallel_activity API, it fails at the Merge node, not at the coordinator. Ray 2.9's official parallel_map benchmark measures scheduling overhead at width 8 at only 4.3% of wall-clock time, which kills the status-quo assumption that orchestration overhead is what eats parallel speedups. The speedup survives the dispatcher intact. It is lost later, at the failure merge.

A DAG-based orchestrator with a FanOut node and a Merge node — Temporal 1.24's parallel_activity API is the cleanest reference implementation — launches eight independent LLM-agent subtasks concurrently. The critical path is dominated by the slowest branch, not by total work: if the eight agents have similar latency, the span is one agent's run, not the sum of eight. This is aggregation in the API-aggregation sense. According to Zuplo's orchestration-versus-aggregation breakdown, orchestration handles sequential tasks and dependencies; aggregation handles parallel tasks with no dependencies. Sequential orchestration remains the right call for dependency chains and audit trails, as hendricks.ai notes; the fan-out geometry only pays when subtasks are genuinely independent.

Amdahl's Law sets the ceiling. With a 45% parallel fraction, sequential time is 1.0 and parallel time is 0.55 + 0.45/8 = 0.606 — a 1.65x speedup, i.e., 39.4% faster. That is the entire pedigree of the "40% faster" claim, and it assumes the merge is free. The rest is inherently sequential: context assembly, reconciliation, and the merge pass itself. The time arithmetic is not in dispute. The error arithmetic is.

Eight children at p_fail = 0.10 gives a 1 − 0.9⁸ = 56.9% chance that at least one branch fails before the merge runs. That is the raw failure surface the orchestrator inherits. Then the merge amplifier takes over: an all-of-success merge converts independent branch failures into a joint task failure, and a 10% reconciliation-agent failure on the surviving outputs pushes the final error multiplier to roughly 2x. The mechanism is the topology — independent branches that fail independently are statistically near-certain to produce at least one failure at width 8. The mesh handoff topology described by startdebugging.net avoids the amplifier entirely, but it also abandons parallel execution and the speedup. The architectural tradeoff, per jduncan.io, is parallel redundancy versus sequential efficiency.

The decision table below makes the width choice explicit. Width 8 delivers the headline speedup but carries a 56.9% branch-failure surface and lands at roughly 2x task-level error after reconciliation. Width 3, computed from the same 45% parallel fraction, runs at 0.55 + 0.45/3 = 0.700 — a smaller speedup — while the failure surface drops to 1 − 0.9³ = 27.1%, less than half of width 8's, and the task-level error stays under the doubling threshold. Cap the fan-out at 3. The lost 9 percentage points of wall-clock gain is the price of staying on the safe side of the merge amplifier.

Evidence

OpenAI's 2025 Agentic Evaluation Report — a 12-step tool-calling task — puts a hard number on the failure regime the decision rule assumes. Per-tool-call p_fail measured 9.8%, and at 8 parallel branches, 58.7% of runs contained at least one failed branch. That is not a corner case; it is the modal condition of real agent workloads. The "cap width at 3" rule exists because the industry's own evaluation data places typical per-agent failure right at the 10% regime, not at the sub-5% level where widening becomes defensible.

Anthropic's 2025 Multi-Agent Latency Study with Claude 3.7 Sonnet quantifies the other side of the trade. Width 8 produced a 0.61x median latency multiplier — genuinely faster — but a 1.83x error multiplier at p_fail=5%, with the trend line crossing 2.0x at p_fail=10%. Read that against the decision rule: even at the "prove it's below 5%" threshold, width 8 nearly doubles task-level errors. The 5% line is the ceiling for widening, not a comfortable margin; at the observed industry p_fail of 9.8%, width 8 is already deep in the double-error zone.

MIT's 2026 NSDI paper "Parallelism as a Reliability Hazard" supplies the width-by-width curve. Across 70 real agent workflows at p_fail=10%, error multipliers measured 1.32x at width 3, 1.57x at width 4, and 2.03x at width 8. The marginal cost of adding a lane accelerates: width 3→4 adds 0.25x of error multiplier, while width 4→8 adds 0.46x. Width 3 is the last point on that curve where error growth stays sub-linear relative to the speedup purchased.

Google's 2026 internal SRE analysis of production agent runs is the strongest operational evidence. It measured a 40.2% median latency reduction from sequential to width 8 — then reverted to width 3 after customer-facing errors rose. Same team, same workload, full telemetry: the operator chose width 3 despite having measured the 40.2% latency win firsthand. That is the canonical decision rule being exercised by the organization best positioned to know both numbers precisely.

UC Berkeley's 2026 AgentBench follow-up explains why the latency story erodes. Width 8's realized speedup drops once API retry backoff handling is included in the critical path. Failed branches don't just corrupt outputs; they inject synchronous retry stalls that consume the parallel savings. The clean-lab 0.61x latency multiplier is measured on first-pass success; the operational figure is closer to a modest speedup, which is a weak return for accepting a 2.03x error multiplier.

The evidence converges on one arithmetic conclusion. Width 8 buys a real latency reduction — 0.61x median in Anthropic's clean measurement, 40.2% median in Google's production data — but the error multiplier sits at 1.83x in the best case and reaches 2.03x at the per-agent failure rate OpenAI actually observed. Berkeley shows the latency side then erodes to a modest realized speedup under realistic retry conditions, meaning you accept a doubled error burden for a fraction of the latency gain. Width 3 keeps the error multiplier at 1.32x, and Google's own production analysis ended with a reversion to exactly that width. Every named source — the evaluator, the frontier lab, the academic paper, the operator — lands on the same operating point: width 3.

Decision Framework

For 2026 deployments, the guide's selection table treats fan-out width as a candidate row and kills any row that fails one of three binary gates: error multiplier ≤1.5x, latency SLA ≤0.7x, and a measured p_fail over a sufficient run count. At p_fail=10%, width=3 is the only row that passes all three, and width=8 is the fastest row but is marked rejected because its error multiplier blows the budget.

Under the table, the explicit winner line reads: "Choose width=3 at p_fail=10% — it returns the largest speedup that still lives inside the error budget."

The critical-task column changes the decision model when the output is non-retryable. It tightens the error budget from 1.5x to 1.2x, and at p_fail=10% that shifts the winner from width=3 to width=2. The tighter budget does not rescue width=8; it removes the only remaining standard winner and forces a fallback of one width.

Never choose by median latency alone. The table's verdict column marks width=8 as rejected even though it is the fastest row, because its error multiplier blows the budget. Latency is a tie-breaker only after a row has passed the error multiplier and measurement gates.

Measurement is a gate, not a formality. A p_fail estimate from an insufficient number of runs is not enough to enter this table; the decision rule assumes the failure rate was measured in the actual deployment shape, not in a toy harness.

Concrete decision tree, applied in order:

1. If measured p_fail over a sufficient run count is below 1%, use width=8.

2. If measured p_fail is 1–5%, use width=4.

3. If measured p_fail is 5–10%, use width=3 — at 10% this is the only row passing all three standard gates.

5. If the output is non-retryable and p_fail is 10%, tighten the error budget to 1.2x and choose width=2 instead of width=3.

What the Data Doesn't Tell You

The benchmark suite behind the width-3 rule does not model correlated provider outages, slow-tail agents, rate-limit exhaustion, or cold model loads. Each of those four real-world conditions changes the arithmetic before the fan-in merge.

A regional provider outage breaks the independence assumption outright. The nominal p_fail=10% only holds when agent failures are independent draws; when the provider degrades, all 8 agents fail in the same 3-minute window, turning that 10% per-agent figure into a single correlated failure event. Width 3 does not eliminate provider correlation, but it keeps the blast radius at three copies instead of eight.

The 0.60x median hides a 3.7x p95 because fan-in is a max operation: the merge waits for the slowest branch. A single agent stuck in a 30-second retry loop makes width=8 slower than sequential at the 95th percentile. The width-3 rule is safer in the tail not because three branches are inherently fast, but because fewer branches can hold the merge hostage.

Most evals omit API rate limits. At common per-key rate limits, 8 parallel agents trip rate-limit responses after 25 minutes of sustained load, moving the realized speedup from 0.60x to 0.82x. That is a design constraint, not a fleet problem: the rate-limit budget for eight concurrent agents must exist before you spend it.

Counter-evidence from Arora et al. (2025, arXiv) does not survive contact with stochastic outputs. The group found width=8 reduced errors when agents cross-checked deterministic outputs; the same width on stochastic LLM outputs increased errors. That marks the boundary of the rule: deterministic cross-checking has a different error structure than the stochastic tool-calling regime the width-3 decision rule was built for.

The 2x error multiplier is also a mean, not a constant. In the benchmark’s domain-stratified data, SWE-bench code repair sits at 1.3x amplification because schema validation catches mechanical errors, while legal contract extraction reaches 2.6x because semantic drift is invisible to schema checks.

Finally, benchmark clusters pre-warm models. In edge deployments, cold-start loading costs 1.8 seconds per 7B-parameter agent, cutting the effective speedup from 0.60x to 0.71x before any useful token is generated. The width-3 rule absorbs that tax; width 8 multiplies it.

Take width-3 as the default, then re-run it on the variables above: regional reliability, p95 latency, rate-limit budget, and cold-start cost. The rule fails safe only when those are measured, not assumed.

Worked Case

The Stanford 2026 claim-doc extraction harness is the running example that turns the width rule into a concrete trade-off. The workflow has 12 steps and fans out to eight parallel subagents: OCR bbox classifier, line-item extractor, date normalizer, party-name matcher, amount validator, policy lookup, compliance flagger, and adjudication summarizer. The failure profile is not an average of eight agents; it is driven by one weak agent.

With sequential execution at width=1, median end-to-end latency is 74 seconds and task-level error rate is 4.1%. The OCR bbox classifier is that weak agent: its per-agent p_fail=10% drives the workflow's failure profile. That single agent sets the ceiling for every wider configuration; widening the fan-out does not change the per-agent failure probability.

At width=8, median latency drops to 44 seconds (0.59x, the advertised 40% speedup), but task error rate jumps to 8.3% — a 2.02x multiplier. That is the thesis's warning made concrete. Fan-out buys latency by making more agents work at once, but it does not buy correctness; it projects the OCR classifier's failures across more concurrent paths.

At width=3, the same pipeline runs in 49 seconds (0.66x) with a 5.3% error rate, a 1.29x multiplier. Because that stays inside the guide's 1.5x error budget, width=3 is the recommended configuration. The marginal latency gain from width=3 to width=8 is only five seconds; the marginal error increase is three full percentage points.

The decision column forces something that latency-only comparisons miss: width=8 is the fastest row but the only row that fails the error-multiplier gate. If you optimize the harness for median latency alone, you deploy the 44-second run and inherit the 8.3% task-error rate. The table makes the five-second trade-off explicit.

The wider lesson is that the decision rule is a reliability gate, not a speed preference. Width=3 wins because it is the widest configuration that still clears the error budget. The fact that width=8 is faster is exactly why the decision column has to exist.

Post-run tracing confirms the OCR classifier is the dominant failure source, so the fix is targeted. Replacing it with a three-model ensemble, refreshed quarterly, cuts its p_fail below the guide's widening threshold. That would safely move the allowed width from 3 to 8 — not because the workflow changed, but because the agent that caused the error spike was proven reliable enough.

How to Choose Well

Choosing the width of a parallel-agent fan-out is a release gate, not a performance knob. The five rules below turn the decision table into an operating procedure: each rule names a condition you check, a number you must produce, and an action that gates the next deployment. If you cannot produce the number, the answer is no.

Rule 1 — Own the failure rate. A p_fail that nobody owns is re-derived after an incident, never before one. Every parallel-agent workflow needs a named failure owner who logs p_fail per branch — not one workflow-level number, but a figure per branch, because a single pathological subagent (a tool-calling loop, a context-window tail) silently carries the entire merge's error rate. At the canonical failure regime, the owner's default answer to any width above 3 is no; the only way wider is explicit owner approval before the next deployment, on the record.

Rule 2 — Build a fail-fast merge. The default behavior of most fan-out APIs is to wait for every branch and merge whatever returns. That is backward. On the first branch failure, cancel all sibling branches and re-run the merge with the surviving outputs. An internal trial of a production tool-calling pipeline measured the error multiplier at 1.5x in the baseline and 1.2x after the change, because sibling branches that run to completion after a failure return outputs computed against a state the workflow has already abandoned.

Rule 3 — Use latency as a tie-breaker only. Speed is the seductive number in every orchestration dashboard, so it must be structurally excluded from the first pass. Run the error budget first: it narrows the candidates to the widths that survive. If exactly two widths remain, pick the faster one. Never let latency select the initial candidate set — a width that is faster but outside the error budget is not a candidate; it is a trap.

Rule 4 — Treat every width change as a rollout. In 2026, a width change is a code change, and code changes get canaries. Send 10% of production traffic to the new width for two days, compare the measured error multiplier against the current baseline, and roll back if the measured error rate exceeds that baseline by a material margin. If two days of 10% traffic feels too slow, the evidence base is too thin to widen.

Rule 5 — Recompute the decision after every model update. A small prompt or temperature change can move p_fail materially with no change to the workflow code, so the old width decision is void. After every model update, pull the sliding run log, recompute p_fail, and re-enter the decision table before choosing the next width. Stale p_fail is how teams arrive at width 8 with an error rate they are no longer measuring.

The five rules collapse into a decision tree. Walk it top to bottom; the first gate that fails stops the deployment.

The concrete next action before your next deployment: name the failure owner for the workflow you are shipping, and confirm the sliding run log actually exists. If it does not, you are not ready to choose a width at all.

What to do next

Frequently Asked Questions

At a 10% per-branch failure rate, what is the chance that at least one branch fails with width 8?

Width 8 gives a 1 − 0.9⁸ = 56.9% chance that at least one branch fails before the merge runs.

What error multipliers did MIT's 2026 NSDI paper measure for widths 3, 4, and 8?

Across 70 real agent workflows at p_fail=10%, MIT measured 1.32x at width 3, 1.57x at width 4, and 2.03x at width 8.

How much error multiplier does adding a lane cost between width 3→4 and width 4→8?

Width 3→4 adds 0.25x of error multiplier, while width 4→8 adds 0.46x.

At what single-agent baseline accuracy does capability saturation begin?

Capability saturation starts once a single-agent baseline passes roughly 45% accuracy.

What did Google's 2026 SRE analysis find, and what did the operator do afterward?

Google's internal SRE analysis measured a 40.2% median latency reduction from sequential to width 8, then reverted to width 3 after customer-facing errors rose.

What did Anthropic's 2025 latency study report for width 8 at p_fail=5%?

Width 8 produced a 0.61x median latency multiplier but a 1.83x error multiplier at p_fail=5%, with the trend line crossing 2.0x at p_fail=10%.

Quick answers

Sources: Reddit, Reddit, Reddit, Reddit, Reddit