# Building a health check system for your AI agent fleet: 3-minute latency budget vs. 15-minute timeout decision

Colton Ramsey · November 9, 2026

> Costs and numbers that change the decision At scale, the cost difference between these two thresholds is substantial. Consider a scenario with 1,000 runs and a

## Costs and numbers that change the decision

At scale, the cost difference between these two thresholds is substantial. Consider a scenario with 1,000 runs and a 1% stuck rate, yielding 10 stuck agents. A 15-minute terminal timeout spends 10 multiplied by 15, or 150 agent-minutes on these stuck runs. In contrast, a 3-minute kill spends 10 multiplied by 3, or 30 agent-minutes. The delta between the two strategies is 120 agent-minutes reclaimed per 1,000-run window. This arithmetic converts the 3/15 split into heartbeat-count and agent-minute calculations, making the case for the tighter probe budget the default choice for cost-conscious fleets.

Grounding these numbers in the 2026 public record, Gamma demonstrates that AI generation completes in 1-2 minutes while total creation, including review, spans 15-30 minutes; these are separate clocks, so a single 3/15 threshold cannot govern all agent fleets. Artificial Analysis Decision Index v2 reports cost per task ranging from $14 per 1M Tasks for Celeris-1.1 Decision to $859 for Clef, illustrating that resource consumption varies by model and task type. These figures confirm that the 3-minute probe with heartbeat-gated 15-minute timeout is not a universal constant but a calibrated default that must be adjusted to the specific cost profile of the agent fleet in operation.

![Costs and numbers that change the decision — Building a health check system for](https://static.mm-ais.com/article-images-ai/building-a-health-check-system-for-your-ai-99d55ce3.jpg)

## Limits: where the timeout rule breaks

| Edge Case | Wait / Condition | Rule Break | Rule Wins |
| --- | --- | --- | --- |
| External batch tool | Wait longer than 3‑minute health‑check budget | Breaks at 3 min | Wins at 15 min only with ≤60 s heartbeat and idempotent retry |
| Human approval (sync) | Synchronous step after health‑check | Breaks both 3 min and 15 min timeouts | Resolved by moving to async state machine |
| No heartbeat / no persisted state | Missing heartbeat or state persistence | 15 min timeout fails (stall indistinguishable) | 3 min terminal kill remains effective |

Human approval performed synchronously creates a bottleneck that can defeat both the 3-minute and 15-minute timeout rules; moving the approval step into an asynchronous state machine removes it from the health-check path. Before citing any outside roll-call training example, verify that the source actually measures synchronous interruptions in your context; the cited Police1 item is about roll-call training, not AI agent health checks.

These edge cases show where the timeout rule breaks: external batch tools may need longer than the 3-minute health-check budget; synchronous human approval can block both the 3-minute and 15-minute rules; and a run without heartbeat or persisted state cannot qualify for the extended timeout. The three-control design—heartbeat, boundary-state persistence, and idempotent retry—keeps the fleet enforcing its health-check budget under those conditions.

![Limits: where the timeout rule breaks — Building a health check system for](https://static.mm-ais.com/article-images-pixabay/building-a-health-check-system-for-your-94d7d029.jpg)

## And copyable health-check sheet

The illustrative fleet of 100 runs establishes the operational envelope for the 3-minute p95 rule. Three agents process the workload, with an 180-second probe budget and a 900-second terminal timeout governing the overall lifecycle. Of the 100 runs, 72 finish under 3 minutes, 8 batch runs consume the full 8-minute tool duration, and 20 runs stall after 6 minutes with no heartbeat signal. This distribution creates the baseline from which the tiered health-check mechanism derives its thresholds.

Checkpoints execute at fixed intervals to enforce the three-tier probe/watch/terminate logic. At the 180-second mark, 72 runs pass and 28 enter the watch window. By 360 seconds, the 8 batch runs transmit a heartbeat and 20 stalled runs have already missed 12 of the expected 30-second beats. At 540 seconds, the batch runs continue their heartbeat stream, while the stalled runs remain silent. These checkpoints are not arbitrary; they are the decision points that trigger the terminal timeout only when the heartbeat gate is satisfied.

The terminal timeout of 15 minutes applies exclusively to runs that emit a heartbeat at least every 60 seconds and persist state at every tool boundary. In the illustrative window, the 8 batch runs meet these conditions and are granted the extended 13 minutes 10 seconds before eventual return. The 20 stalled runs, lacking both a heartbeat and persisted state, are terminated well before the 900-second cap, freeing resources for the 72 completing runs. This distinction proves that the 3/15 split is not a fixed constant but a conditional gate dependent on runtime signals.

![And copyable health-check sheet — Building a health check system for](https://static.mm-ais.com/article-images-pixabay/building-a-health-check-system-for-your-6cdbe688.jpg)

## Decision rules for your agent fleet

The decision logic relies on concrete measurements taken over 100-run windows. Operators should monitor health-check p95 across these windows; if p95 exceeds 3 minutes in two consecutive windows, alert and investigate latency, and do not grant the 15-minute terminal timeout unless the run meets all three controls. If a run satisfies heartbeat at or under 60 seconds, state persisted at every tool boundary, and idempotent retry, the 15-minute timeout becomes an available option for that qualifying run. Evaluate the controls per run; the article does not establish a fleet-wide uniformity rule beyond those measurements.

## What to do next

| Step | Action | Why it matters |
| --- | --- | --- |
| 1 | Baseline the fleet: measure health-check p95 across 100-run windows and record the current value against the 3-minute default. | The 3-minute p95 budget is only enforceable against a measured baseline; the 100-run window is the unit of every downstream decision. |
| 2 | Configure the alert to fire only when p95 exceeds 3 minutes in two consecutive 100-run windows. | Two consecutive windows separate sustained degradation from a one-off spike, preventing false pages. |
| 3 | Verify each agent emits a heartbeat at least every 60 seconds before granting the 15-minute terminal timeout. | Heartbeat interval at or under 60 seconds is the first of the three mandatory controls. |
| 4 | Persist state at every tool boundary and audit the writes in staging. | Per-boundary persistence is required alongside heartbeat and idempotent retry; a gap here disqualifies the extended timeout. |
| 5 | Make every tool retry idempotent and prove it with a forced double-execution test. | Without idempotent retry, the 15-minute terminal timeout does not apply — full stop. |
| 6 | If any of the three controls is false, hard-kill the agent at 3 minutes instead of waiting for the 15-minute timeout. | All three controls must be present; otherwise the extended timeout risks unsafe retries on corrupted state. |

## Frequently Asked Questions

**What is the default timeout threshold for the 3-minute probe?**

The 3-minute probe with heartbeat-gated 15-minute timeout is the calibrated default.

**How many agent-minutes are reclaimed per 1,000-run window by choosing a 3-minute kill over a 15-minute timeout?**

120 agent-minutes reclaimed per 1,000-run window.

**What is the cost per task for Celeris-1.1 Decision according to Artificial Analysis Decision Index v2?**

$14 per 1M Tasks for Celeris-1.1 Decision.

**What is the cost per task for Clef according to Artificial Analysis Decision Index v2?**

$859 per 1M Tasks for Clef.

**What is the AI generation time range that grounds the 2026 public record figures?**

AI generation completes in 1-2 minutes.

**What is the total creation time range including review that grounds the 2026 public record figures?**

total creation, including review, spans 15-30 minutes

## Quick answers

| What is the agent-minute cost difference between a 15-minute timeout and a 3-minute kill for 1,000 runs with a 1% stuck rate? | 120 agent-minutes reclaimed per 1,000-run window |
| --- | --- |
| How does the 3-minute latency budget relate to the 15-minute timeout in the context of AI generation and review clocks? | AI generation completes in 1-2 minutes while total creation, including review, spans 15-30 minutes; these are separate clocks, so a single 3/15 threshold cannot govern all agent fleets. |
| What does the Artificial Analysis Decision Index v2 report about cost per task for different models? | cost per task ranging from $14 per 1M Tasks for Celeris-1.1 Decision to $859 for Clef |
| Why is the 3-minute probe with heartbeat-gated 15-minute timeout considered a calibrated default rather than a universal constant? | resource consumption varies by model and task type, confirming that the threshold must be adjusted to the specific cost profile of the agent fleet in operation. |
| What is the primary reason for choosing a 3-minute latency budget over a 15-minute timeout in cost-conscious AI agent fleets? | The 3-minute kill strategy reclaims 120 agent-minutes per 1,000-run window compared to a 15-minute timeout, making it the default choice for cost efficiency. |

Also worth reading: **Audit and trace AI agent decision chains**: [Audit and trace AI agent](https://tryinterlock.com/blog/audit_and_trace_ai_agent_decision_chains.php) · **Human-in-the-loop agent workflows: 7 best practices that scale**: [Human-in-the-loop agent workflows: 7 best](https://tryinterlock.com/blog/human_in_the_loop_agent_workflows_7_best_practices_that_scale.php) · **Human-in-the-loop approvals for critical AI agent decisions**: [Human-in-the-loop approvals for critical AI](https://tryinterlock.com/blog/human_in_the_loop_approvals_for_critical_ai_agent_decisions.php)

### Related reading

- [Building Fault-Tolerant AI Agent Workflows That Resist Failure](https://tryinterlock.com/blog/building_fault_tolerant_ai_agent_workflows_that_resist_failure.php)
- [Stop Cascading Agent Failures: 3 Retries, 2s Timeout, 50% Breaker or Fail Fast?](https://tryinterlock.com/blog/stop-cascading-agent-failures-3-retries-2s-timeout-50-breaker-or-fail-fast.php)
- [Migrate agent workflow: migrate if 95th percentile (p95) latency exceeds 2s across 3 tools](https://tryinterlock.com/blog/migrate-agent-workflow-migrate-if-95th-percentile-p95-latency-exceeds-2s-across-3-tools.php)
- [5-Agent Pipelines: OpenAI SDK vs LangGraph Handoff Latency](https://tryinterlock.com/blog/5-agent-pipelines-openai-sdk-vs-langgraph-handoff-latency.php)
- [Multi-Agent Deadlock: Recovery Strategy Cuts Latency 30%](https://tryinterlock.com/blog/multi-agent-deadlock-recovery-strategy-cuts-latency-30.php)
- [RAG vs Fine-Tune 2026: Latency Down 30%, Accuracy Up 2%](https://tryinterlock.com/blog/rag-vs-fine-tune-2026-latency-down-30-accuracy-up-2.php)

### Latest

- [How To Limit Memory: General Data Protection Law Article 5 Delete or Cut 80%...](https://tryinterlock.com/blog/how-to-limit-memory-general-data-protection-law-article-5-delete-or-cut-80-token-costs.php)
- [Agent Workflows in 2026: When to Enforce a Formal Orchestration Contract](https://tryinterlock.com/blog/agent-workflows-in-2026-when-to-enforce-a-formal-orchestration-contract.php)
- [Pinpoint a single agent decision in 2026: 3-Hop Trace Limit to Verify in...](https://tryinterlock.com/blog/pinpoint-a-single-agent-decision-in-2026-3-hop-trace-limit-to-verify-in-production.php)

Canonical: https://tryinterlock.com/blog/building-a-health-check-system-for-your-ai-agent-fleet-3-minute-latency-budget-vs-15-minute-timeout-decision.php
Markdown: https://tryinterlock.com/blog/building-a-health-check-system-for-your-ai-agent-fleet-3-minute-latency-budget-vs-15-minute-timeout-decision.php/index.md