Case Studies

Amazon Split Claude’s Role to Cut Agent Costs

Amazon says replacing Claude Sonnet 4.6 with Strands Decider saved millions of dollars. The gains came from separating an Agent’s decision layer from content generation, allowing each layer to use a more appropriate model.

Font size

Amazon says replacing Claude Sonnet 4.6 with Strands Decider saved millions of dollars. The more important lesson is not the lower model price, but the fact that the Agent’s decision layer was optimized independently.

Where the Millions Were Saved

Amazon says Strands Decider saved millions of dollars when it replaced Claude Sonnet 4.6 in a production workload.

The public summary says the change involved more than selecting a different model. It also affected inference costs, the Agent’s decision framework, and model-routing strategy.

These savings come from two sources:

  • Lower inference costs: A less expensive model handles each request, reducing the cost of both input and output tokens.
  • Lower task-chain costs: Less repeated reasoning, fewer invalid tool calls, fewer routing errors, and fewer retries.

The millions of dollars were a business result reported by Amazon. However, the original case study did not disclose the full calculation period, workload volume, concurrency, or itemized costs before and after the switch.

That makes it difficult to attribute the entire saving to the model’s unit price.

A more accurate interpretation is that Strands Decider changed the Agent’s decision layer. Some work previously handled by Claude Sonnet 4.6 may now be routed elsewhere, cached, or assigned to a model better suited to structured decisions.

Sonnet Generates; Decider Decides

The difference between Strands Decider and Claude Sonnet 4.6 is not that one ranks above the other on capability benchmarks. They serve different purposes.

The architecture has two layers:

  1. Generation layer: Understands the request, organizes the answer, and produces code or tool-call parameters.
  2. Decision layer: Determines the next action, whether to call a tool, and which path to follow.

Traditionally, one general-purpose model handles all of these tasks.

If a task requires eight reasoning turns, three tool calls, and two retries, an expensive model may be invoked repeatedly. The context also tends to grow with each turn.

总推理成本 ≈ Σ(每轮输入 Token × 输入单价
               + 每轮输出 Token × 输出单价
               + 工具调用与重试附加成本)

If a smaller model can handle decisions and produce shorter outputs, Sonnet 4.6 does not need to process every step.

This is not a weaker model replacing a stronger one. It is the separation of routing decisions from content generation. Sonnet 4.6 continues to handle understanding and generation, while Decider handles structured judgments.

The approach resembles a compiler:

  • The intermediate representation is stable enough.
  • Different stages can use different models.
  • Decisions can be recorded, evaluated, and replayed.
  • Expensive models run only on necessary paths.

The Counterintuitive Case Against Always Using Expensive Models

When selecting a model, companies commonly estimate token costs for a single request:

estimated_cost = (
    input_tokens / 1_000_000 * input_price
    + output_tokens / 1_000_000 * output_price
)

But an Agent’s actual bill includes more than tokens. One request may trigger a multistep workflow:

用户请求
  → 意图判断
  → 工具选择
  → 参数生成
  → 工具执行
  → 结果解析
  → 失败重试
  → 最终回答

A single routing error can make every expensive call that follows it unnecessary.

If a cheap model selects the wrong tool repeatedly, it lengthens the task chain. If an expensive model handles every minor decision, token usage also accumulates quickly.

Optimizing an individual generation does not necessarily reduce the cost of the entire task chain.

More useful metrics include:

  • Total model cost per successful task.
  • Number of tool calls per successful task.
  • Share of invalid steps.
  • Retry rate and recovery rate.
  • Whether decisions can be reproduced consistently.

Agent costs are better treated as a compute budget than as a model price sheet.

Four Costs Enterprises Should Account For

1. Model Costs

These include input and output tokens, cache hits, tool results written back into the context, and additional tokens consumed by retries.

If Decider produces short JSON, it can theoretically reduce the number of times an expensive model handles structured decisions. Whether costs actually fall depends on routing accuracy, not Decider’s per-call price alone.

2. Orchestration Overhead

An Agent also incurs costs for state management, context assembly, event dispatch, tool-protocol conversion, and timeout control.

These costs may not appear on the model’s token bill, but they still affect latency and maintenance effort.

3. Retry Mechanisms

Tool failures, invalid outputs, and timeouts can all trigger retries. Retries should follow explicit rules rather than resend the same context to an expensive model and hope for a better result.

Common practices include:

  • Retrying recoverable errors a limited number of times.
  • Mapping error types to fixed recovery paths.
  • Validating decision outputs against a schema.
  • Stopping a task after repeated failures to avoid further token consumption.

4. Evaluation

Looking only at average token cost can lead enterprises to the wrong conclusion that the cheapest model produces the best system.

A more useful evaluation segments results by task outcome and records at least the following:

task_success_rate
tool_call_count
invalid_tool_call_rate
retry_rate
end_to_end_cost
p50 / p95 latency

Teams can then compare two architectures:

  • Claude Sonnet 4.6 handles every step.
  • Sonnet 4.6 handles generation, while Strands Decider handles routing and decisions.

The switch makes sense only if task-success rates remain comparable while total costs fall.

Where the Case Study Falls Short

After the original article was published, it reached the front page of Hacker News. Discussion focused on three questions.

Can the Benchmark Results Be Reproduced?

A business case and a public benchmark are not the same thing.

Amazon’s workload includes internal data, tools, prompts, and routing rules. External teams testing Strands Decider with simulated tasks cannot reproduce the full cost structure.

Validating the conclusion requires more than final average cost. It also requires workload distribution, success rates, and retry data. Without them, “saving millions” is better understood as a case-study result than a universal model-selection rule.

Can the Architecture Move to Other Enterprises?

The architectural idea can transfer, but the specific numbers cannot be copied directly.

Optimizing the decision layer independently is usually valuable when tasks are highly structured, tool calls are frequent, and there are only a limited number of possible paths. If tasks depend on open-ended reasoning, decision and generation costs remain intertwined. Splitting the layers may then increase orchestration overhead and even reduce success rates.

Is the Millions-of-Dollars Figure Transparent?

The available material does not establish:

  • Whether the savings cover several months or a full year.
  • The scale of the workload.
  • Whether migration, monitoring, and evaluation costs are included.
  • Whether the savings came from lower model prices, fewer calls, or both.
  • The model and infrastructure costs used to run Decider itself.

This does not invalidate the case. It does mean enterprises should examine its limits rather than simply copy its conclusion.

Editorial Perspective

The important part of this case is not that Amazon found a cheaper model. It is that Amazon separated the Agent’s decision about what to do from its work of actually doing it. This differs from the more familiar approaches to reducing model costs. Distillation, quantization, caching, and smaller models primarily lower the cost of individual inferences. This case redistributes the compute budget across an entire task chain.

Model routing is likely to become a standard component of enterprise Agents. Decider, however, may not remain tied to a particular model indefinitely. It is better understood as an evaluable, replaceable decision interface. Amazon may also change its underlying model while preserving the boundary between the decision and generation layers.

The uncertainty is equally clear. The available material does not fully explain the period, task volume, or migration costs covered by the reported savings. Publishing total savings without success rates, call counts, and retry rates can be misleading. The transferable lesson is not to copy the dollar figure, but to account for the full cost by task outcome and then decide which layer warrants an expensive model.

Source

Source: https://strandsagents.com/blog/amazon-saves-millions-replacing-sonnet-with-strands-decider/