Measuring retry amplification in AI agent workflows

An AI agent can finish one task after one model response, or it can finish the same task after three model responses, four tool calls, a timeout, and a repair pass. Both runs may appear as one completed job in a dashboard. They are not the same workload.

Some extra work is necessary: reading a log and then querying a metric are different operations. A retry repeats an operation because the earlier attempt failed, was rejected, or left an uncertain result. Measuring retry amplification requires separating those repeats from the workflow’s intended steps. Otherwise a useful multi-step investigation can look like a retry problem.

Preserve one logical task ID for the requested outcome, an operation ID for each intended model or tool operation, and a fresh attempt ID each time that operation is tried. Then report retry load alongside cost and latency per accepted outcome, keeping failed and unknown work visible.

Agent cost becomes useful operational evidence when an attempted task can be connected to its accepted outcome. Evaluate attribution across retries, rejected results, and correction work before treating a lower token bill as an improvement.

Count logical work and physical attempts separately

A logical task ID follows the user’s requested outcome across the whole run. Within it, an operation ID identifies a particular model request or tool action, while an attempt ID identifies one execution of that operation. Retrying the same tool action keeps its operation ID and changes its attempt ID. A later, different tool action gets a new operation ID. Resuming a turn is not itself evidence of a retry; inspect which operations it actually repeats.

This is a familiar tracing distinction. OpenTelemetry’s guidance for writing semantic conventions asks authors to specify whether a client span represents a logical call or a physical per-attempt call. Its generative AI attributes also define conversation and tool-call identifiers. Do not assume that every span is an attempt: establish what the instrumentation records, then connect that evidence to the task and operation identifiers in your application.

OpenAI's Agents tracing documentation groups model responses and tool calls beneath a turn and records duration, status, arguments, and results when available. That trace can show the physical work. Your application still has to decide which result was accepted by the consumer.

Use two ratios, not one success rate

For a task designed to contain one logical operation, attempts per task is a useful first ratio. Divide all physical attempts by all eligible tasks. If every task reached its first attempt, 1.00 means none needed a repeat; 1.27 means there were 27 percent more attempts than tasks. For a multi-operation workflow, use attempts per logical operation to isolate retry amplification, and retain attempts per task as a measure of total work.

The second ratio is attempts per accepted outcome. Divide the same attempt count by outcomes that passed the consumer's acceptance check. This ratio grows when tasks fail, end in an unknown state, or produce results that require correction. It exposes work that a simple completion rate can hide.

Consider a hypothetical batch of 100 support-triage tasks, each requiring one classification operation and reaching at least its first attempt. Seventy-eight succeed on the first attempt. Fourteen succeed after one retry. Five fail after three attempts, and three stop after two attempts with an unresolved destination state. The workflow made 127 attempts, so retry amplification is 1.27 attempts per operation and, in this example, per task. It produced 92 accepted outcomes, so it required about 1.38 attempts per accepted outcome. The three unknown results do not count as either success or failure until the destination is reconciled.

Keep model generations and external tool attempts available as separate dimensions. A reasoning retry consumes different capacity from a repeated database write. If they are collapsed into one counter, the total may explain a cost increase without explaining the operational risk.

Connect cost and latency to the accepted outcome

Once task, operation, and attempt records are connected, calculate the monetary cost of every attempt. Use the provider’s non-overlapping billed token categories and their applicable rates; do not add cached-token or reasoning-token subtotals again when they are already included in an input or output total. Add tool fees and the monetary cost of execution time, then divide the batch’s total cost by its accepted outcomes. Keep the costs of failed and unknown tasks in that total.

A cheaper model call can make the complete workflow more expensive if it causes more tool retries or correction work. A slower first attempt can be the better choice if it produces a result the consumer accepts without another pass. Neither conclusion is universal. The point is to compare the complete task under the same acceptance rule.

Latency needs the same treatment. Report user-visible task latency alongside time spent in generations, tool calls, backoff, queues, and reconciliation. Parallel work should not be added as if it were sequential elapsed time. The trace describes where work occurred; the task clock describes how long the user waited.

Treat unknown outcomes as their own state

A timeout is not proof that an action failed. The destination may have accepted the change before the acknowledgment was lost. OpenAI's agent recovery guidance tells clients to check saved work and completed actions before retrying a failed turn. It also recommends honoring retry limits and stopping when the error changes.

That recovery behavior should appear in the metric. Record an outcome as unknown when the caller cannot establish whether the side effect occurred. Reconcile it through the destination's read path, an idempotency key, or a stable external identifier. Only then move it to accepted or failed.

Unknown work should also have an age. An unresolved write that is ten seconds old may still be inside a normal reconciliation window. The same state after thirty minutes may need an operator. Count and alert on unknown outcomes by age rather than forcing them into a success percentage.

Build the dashboard around decisions

A useful operating view begins with eligible logical tasks, accepted outcomes, explicit failures, and unknown outcomes. Add attempts per logical operation to show retry load, and keep attempts per task as a measure of total work. Also show attempts per accepted outcome, task latency, cost per accepted outcome, and the distribution of attempts by model, tool, and error class. Keep retries that were prevented by idempotency visible because they still consumed work even though they avoided a duplicate side effect.

Segment before blaming the model. A rising amplification ratio may come from a provider overload, a slow MCP server, invalid tool arguments, a destination conflict, or an application acceptance check that rejects otherwise completed turns. Each cause has a different owner and a different repair.

The boundary is equally important. When the agent platform does not expose physical attempts or stable task identifiers, you cannot calculate retry amplification precisely from completion notifications. Instrument the application and tool boundary you control, mark the remaining work unknown, and avoid presenting the partial ratio as complete.

The prompt-cache SLI guide shows why missing usage cannot be treated as zero. The scheduled execution contract explains how to reconcile a consequential run before retrying it. Retry amplification connects those ideas to one operating question: how much work did the system perform for each outcome the consumer could actually accept?