LLMs

How to Recalculate Agent Costs After 80% LLM Price Cuts

Lower API prices do not automatically make AI Agents cheaper to operate. A practical cost audit must account for multi-turn calls, tool loops, retries, long context, and the cost of each successfully completed task.

Font size

OpenAI and Anthropic have cut the prices of some LLM APIs by as much as 80%. For AI Agents already in production, the first step is not necessarily to switch models. Buyers should recalculate the total cost of each successful task before choosing a replacement.

How to Recalculate Agent Costs After 80% LLM Price Cuts

Over the past few months, OpenAI and Anthropic have reduced the prices of some LLM APIs by up to 80%.

These changes were reported in a Hacker News front-page story titled “OpenAI and Anthropic cut their LLM prices by up to 80% within a few months. Why?”. The HN page showed 12 points and 7 comments.

The prices have fallen, but the first priority for a production AI Agent may not be an immediate model switch. It may be time to reopen the call ledger and rebuild the cost model from the ground up.

Cheaper tokens per call do not necessarily translate into cheaper tasks. Multi-turn calls, tool loops, failed retries, and long context can still consume the budget behind the scenes.

Cheaper Prices Make Agent Cost Accounting More Complex

An Agent's cost cannot be assessed solely by the price per million input and output tokens.

A typical workflow might look like this:

  1. Read the user's request.
  2. Call the model to determine the next action.
  3. Call a search engine, database, or MCP tool.
  4. Add the tool results to the context.
  5. Continue model reasoning.
  6. Generate an invalid response and trigger a retry.
  7. Fail again, then escalate to a human or switch models.

If every turn carries all previous tool results, the context continues to expand. The number of model calls may also exceed the number of requests visible to the user.

Suppose a task averages eight model calls, with 12,000 input tokens and 2,000 output tokens per call. It actually consumes roughly 96,000 input tokens and 16,000 output tokens. If one failure causes the entire sequence to restart, the increase is not limited to 12.5%. If the retry continues to include the full history, the increase could be even larger.

Price cuts primarily improve three areas of spending:

  • Multi-turn calls: A task can now afford more reasoning steps.
  • Long-context processing: Larger codebases, documents, and message histories place less pressure on the budget.
  • Batch inference: Tasks such as offline summarization, classification, and document processing are more practical on lower-priced APIs.

High-frequency retries, tool loops, and expanding context remain significant. Even with an 80% reduction in unit prices, an Agent that originally required 20 calls can consume the savings because of poor routing or prompt design.

Price cuts do not invalidate the cost model. They make its weak points more visible.

Where Did the 80% Price Cuts Come From?

The original article, “Cheap LLMs,” was published on minaxir.com in October 2026:

https://minimaxir.com/2026/10/cheap-llms/

Its central argument is not that inference costs naturally decline. Instead, it points to a combination of factors: improved inference efficiency, higher hardware utilization, model distillation, open-weight models catching up, and vendors using subsidies to suppress API prices.

These factors should be considered separately.

Inference efficiency means that a model requires less computation to produce the same answer. Sparse Transformers, better KV cache management, speculative decoding, and structured outputs can all reduce latency or computation.

Hardware utilization is more closely related to cloud operations. Whether GPU clusters run at high occupancy, whether requests are distributed across batches of different lengths, and whether inference services operate continuously all affect unit costs.

Model distillation allows smaller models to learn the output distribution of larger models. On narrow tasks such as classification, extraction, and rewriting, buyers can trade some accuracy for a lower price. This may not work for complex planning, but it is well suited to localized decisions within an Agent workflow.

Open-weight models catching up adds pressure on pricing. Models such as Qwen, DeepSeek, Llama, and Mistral continue to reduce the premium charged for commercial APIs. Even when self-hosting requires GPUs, operations, and engineering effort, buyers gain a genuine alternative: they can at least move some traffic to an internal cluster.

Vendor subsidies are the easiest factor to overlook. A listed API price does not necessarily reflect a sustainable long-term price. New model launches, developer acquisition, enterprise contracts, and periods of capacity expansion may all come with temporary discounts.

For that reason, when encountering an “80% price cut,” I would first ask four questions:

  • Which model and which input or output tier received the reduction?
  • Are there conditions, rate limits, or cache-write requirements attached to the new price?
  • Have latency, throughput, and availability changed as well?
  • Can this rate be used in budget assumptions for the next 12 months?

Treating a promotion as a structural cost reduction is the most common mistake in cost modeling.

Open Models Catch Up, and Closed Vendors Lose Their Premium First

A commercial API offers more than model capability. It also provides automatic scaling, global availability, security and compliance, tool-calling compatibility, and operations-free infrastructure.

But when the price gap between open-weight models and closed APIs narrows, companies reassess vendor lock-in. Migrating models once required changing prompts, rerunning regression tests, and sometimes modifying tool protocols. Every price reduction lowers the psychological barrier to that migration effort.

In production, model routing matters more than simply choosing a vendor.

A router, for example, could use a low-cost model for intent classification. Simple extraction tasks could run on a distilled model; code-modification steps could use a more capable model; and large volumes of tool-generated logs could be compressed before a decision is made about retaining the full text. Models can be switched after a failure, but retries must be capped.

This is where stateful workflow frameworks such as LangGraph become more valuable. Workflow nodes, state persistence, failure fallbacks, and human handoffs all require explicit design. Changing the model without changing the state machine, tool arguments, or error branches may lower the line-item price without lowering the cost per task.

Open models do not need to replace every closed API. A more realistic goal is to move 30% to 60% of stable, standardized traffic to lower-cost models while retaining closed models for high-difficulty tasks and preserving a rapid rollback path.

What to Change in an Agent Deployment

I would rerun the token cost model before changing any code.

Start by breaking tasks down by success rate, average call count, average context length, and human-intervention rate. At minimum, track five metrics:

  • Model cost per successful task.
  • Total calls per successful task.
  • Tool-call success rate.
  • P50 and P95 latency.
  • Human-handoff rate.

Then recalculate the costs using real traces rather than theoretical token estimates.

The following Python script reads Agent traces, calculates model costs for each successful task, and prints the 10 most expensive tasks. price_in and price_out are expressed in dollars per million tokens:

import json
from pathlib import Path

PRICE_IN = 0.25
PRICE_OUT = 1.25

records = []
for path in Path("traces").glob("*.json"):
    with path.open(encoding="utf-8") as f:
        trace = json.load(f)

    if not trace.get("success"):
        continue

    input_tokens = 0
    output_tokens = 0
    calls = 0

    for event in trace.get("events", []):
        if event.get("type") != "model_call":
            continue
        usage = event.get("usage", {})
        input_tokens += int(usage.get("input_tokens", 0))
        output_tokens += int(usage.get("output_tokens", 0))
        calls += 1

    cost = (
        input_tokens * PRICE_IN / 1_000_000
        + output_tokens * PRICE_OUT / 1_000_000
    )
    records.append({
        "task_id": trace.get("task_id", path.stem),
        "calls": calls,
        "input_tokens": input_tokens,
        "output_tokens": output_tokens,
        "cost_usd": round(cost, 6),
    })

for record in sorted(
    records, key=lambda item: item["cost_usd"], reverse=True
)[:10]:
    print(json.dumps(record, ensure_ascii=False))

After running it, make four changes.

First, route tasks dynamically by difficulty. Do not assign models by brand; route them using task characteristics and evaluation scores.

Second, add caching. Cache stable system prompts, retrieved documents, and repeated tool results. Semantic caching should have a similarity threshold and a tracked hit rate to prevent incorrect reuse.

Third, cap failure retries. Two retries are usually easier to control financially than unlimited retries. Exponential backoff can reduce repeated requests caused by short-term congestion.

Fourth, prepare for human intervention. Critical write, payment, deletion, and publishing operations should not be retried indefinitely by an Agent. Escalation conditions can include two consecutive tool-argument validation failures or a per-task cost that exceeds the budget limit.

Finally, add an interface layer for vendor switching. Keep model names, parameters, timeouts, and retry policies in configuration rather than hard-coding an SDK in core business logic. Otherwise, every price cut requires another round of engineering changes.

The Real Competitive Metric May Not Be Price per Million Tokens

The cheapest model may not be suitable for a critical Agent workflow.

Suppose Model A is 80% cheaper per call, but its tool-call success rate is five percentage points lower and tasks average 0.4 additional retries. Model B is more expensive, but it completes more tasks on the first attempt. The cost per successful task may ultimately be higher for Model A.

Agents should compare the total cost of completion:

Cost per successful task =
Model call costs
+ Tool costs
+ Retry costs
+ Human handling costs
+ Business losses from failures

Latency also matters. If an interactive Agent's P95 rises from two seconds to eight, users may abandon the task. Batch workloads, by contrast, place greater emphasis on throughput and cost per unit of time.

A larger context window is not always better. Increasing the window from 128K to 1M does not necessarily reduce the bill. Many models apply a higher price tier to long inputs. Carrying every piece of irrelevant history consumes tokens and adds noise.

I would retain a small regression set for every model upgrade, covering:

  • Argument generation.
  • Tool selection.
  • Multi-turn state retention.
  • Timeout and error recovery.
  • Prompt injection and unauthorized writes.

Record price, latency, success rate, and cost per completed task at the same time. Price per million tokens alone cannot determine which model an Agent should use.

Editorial Perspective

This round of price cuts reduces the cost of experimenting with Agents rather than simply making models cheaper. Multi-turn reasoning, retries, and long context previously limited the range of viable Agent deployments. With lower unit prices, companies will be more willing to test complex workflows, but that will also amplify inefficient loops. In the short term, teams should audit traces, caching, and rollback plans before deciding whether to migrate models.

One important difference is that price competition is no longer limited to OpenAI and Anthropic. Open-weight models continue to erode the premium, and price volatility has entered the routing architecture. The winning system will not necessarily offer the lowest rate. It will select models dynamically by task and accurately account for failure rates, human handoffs, and vendor switching. However, subsidy terms, contract pricing, and real-world throughput remain uncertain, so a one-time promotional rate should not be used to rewrite the entire annual budget.

Source

https://minimaxir.com/2026/10/cheap-llms/

Source: https://minimaxir.com/2026/10/cheap-llms/