Skip to main content
Agentic Architecture Skills.
Start hereUse casesArchitectureIntelligence ladderKnowledge & memoryResearchSkills
Advanced design and research

Token efficiency: spend tokens on outcomes

An agent's bill grows with every loop. Measure it per resolved outcome, meter four numbers for each agent, and stop the spend before the call.

By Murali SidLinkedInPublished

An agent's bill grows with every loop. AI models are billed by the token, a piece of text about three quarters of a word long. At every step, an agent sends its whole history back to the model. So it uses far more tokens than a chat. Count tokens and cost per resolved outcome, not per call or per seat. Then stop the spend before the next call, not with an alert after it.

Where the tokens go

A chat is one question and one answer. An agent works in a loop. It reads the task, calls a tool, reads the result and decides the next step. Each step is a new call to the model. Each call carries the instructions, the tool descriptions and everything that has happened so far. So the model reads the same context many times. You pay for every reading.

The published multiples come from vendors measuring their own products:

  • A single agent used about 4 times the tokens of a chat. A multi-agent system used about 15 times (Anthropic, 13 June 2025) [vendor]. Anthropic says multi-agent work pays only when the task is worth a lot.
  • Claude Code agent teams use about 7 times the tokens of a standard session. Claude Code averages about USD 13 per developer per active day (Claude Code docs, read 14 October 2026) [vendor]. That is a cost per seat, not per outcome.
  • One production agent reads about 100 input tokens for every token it writes (Manus, 18 July 2025) [vendor]. Manus calls the cache hit rate the most important number for a production agent.

The last figure explains the bill. The output is small. Most of the cost is context read again on every step. Two levers matter most. One is how much context each step carries. The other is how much of it comes from a cache.

The unit: cost per resolved outcome

A cost per call hides retries and failed runs. A cost per seat hides how much work got done. Neither says whether the agent was worth running. This guide uses one unit for that decision, the cost per resolved outcome. It has one definition across the guide, recorded in the decision log. Measure value and full cost gives it in full. For the period, add up three costs:

  • all model, tool and platform cost, failed and retried attempts included;
  • supervision time at a loaded rate;
  • the cost of fixing wrong outcomes.

Divide the total by the outcomes accepted as resolved under the agreed quality standard.

Report the resolution rate, failure rate, serious errors and repeat contacts beside it. Divide by resolution, never by containment. A conversation that never reached a person is not always a problem solved.

Keep two levels, as the economics model does. Cost per run reconciles to the invoice. Cost per resolved outcome decides the business case.

Tokens follow the same rule. Tokens per resolved outcome counts every token in the period, failed and retried runs included. It divides by the same resolved outcomes. Make it the main engineering meter for each agent. An engineer can change it directly. The cost figure also depends on prices and on people's time.

What to meter for each agent

Meter four numbers for every agent. Key each one to the agent's registry entry and its identity. Then the cost lands with the right owner. The traces they come from are covered in the observability and FinOps layer (R12).

Watch the trend for each agent, not a single reading. A new model, a new tool or a longer prompt can move all four. Treat that as a change trigger and measure again before release.

Techniques with a measured effect

Each row is one team's test on a task it chose. The first three are vendors testing their own features. Read them as direction. None gives a percentage you can carry to your own workload.

Caching lowers the price of re-read context. It does not remove context the step does not need. Cut that first. Then cache the part that stays the same, such as instructions and tool descriptions. Keep it at the start of the prompt. A cache only matches the start of a prompt. In Claude Code, a cache entry lasts 5 minutes by default on an API key (Claude Code docs) [vendor]. After a longer pause, the whole context is billed at the full price again.

Check quality on the same test set after every saving. A saving that lowers the resolution rate can raise the cost per resolved outcome.

Budgets tied to the registry

An alert after the call is not a stop. The money is already spent. A budget has to stop the next call before it is made.

EY calls these "agentic circuit breakers" (EY, 1 June 2026). They are spend ceilings, call caps and automatic shut-offs. They work at agent, workflow and business-unit level. EY says to measure the cost per task first, then set the limits.

Model gateways set budgets per key, per user and per team. They have no built-in budget per agent. So teams issue one key per agent (LiteLLM docs, read 14 October 2026) [vendor]. LiteLLM also needs its database running before it enforces any budget. Test that the limit really stops a call.

Tie each key to one registry entry:

  • No key without a registry entry and a named sponsor.
  • The key carries the agent budget, with a hard cap per run and per period.
  • The gateway checks the cap before each call. When the budget runs out, the work goes to a human queue. It does not retry quietly.
  • Retiring the agent revokes its key. See the agent lifecycle.

Every token then traces to one agent, one owner and one purpose.

What nobody has measured

No independent source publishes cost per resolved outcome across real enterprises. Nobody publishes how many tokens a typical enterprise task uses either. The public numbers are vendor tests, benchmark runs and surveys.

The surveys measure confidence, not cost. In one survey, 39 percent of firms could not tie AI spend to an outcome their finance head would accept. The answers are self-reported (State of Tokenomics, 23 September 2026). LangChain's survey of 1,340 people ranks quality ahead of cost as a barrier (LangChain, 12 June 2026) [vendor].

Gartner forecasts that the inference cost of an agentic workflow will rise more than fivefold through 2028. Only search excerpts that quote the release were read (Gartner, 17 August 2026). Both trends can hold at once. The price per token falls while the tokens per workflow rise.

HAL reports totals for full benchmark runs, not a cost per task. EY's example of USD 0.04 rising to USD 1.20 per customer interaction is an illustration, not a measurement. So there is no published figure for what good looks like. Set your own baseline from your own traces.

The incidents, corrected

Two cost incidents are quoted often. Both are often told wrongly.

The USD 1.3 million bill. A three-person team ran a fleet of about 100 coding-agent instances on purpose. Over 30 days the dashboard showed USD 1,305,088.81 and 603 billion tokens. Most of the cost came from the premium Codex Fast Mode price. The project's maintainer said the raw cost would be about USD 300,000 without it. His employer, OpenAI, covers the bill (Tom's Hardware). It was not a runaway loop. The lesson is about a pricing choice and a missing per-agent budget.

The USD 47K loop. In the story, four agents were built with LangChain. Two of them, an analyser and a verifier, passed requests back and forth for about 11 days. They produced nothing usable. No agent had its own budget cap. The whole story traces to one first-person account. An incident catalogue marks it as only partly verified. The weekly figures in circulation do not add up (dev.to analysis). Treat it as low confidence. If it happened as told, the lesson is the same: an alert is not a stop.

Evidence and limits

The multiples, prices and technique savings on this page are vendor-published and marked [vendor]. RouteLLM and HAL are research results on public benchmarks. The surveys are self-reported. The Gartner forecast was read through search excerpts. Prices and cache rules change often. Each one carries the date it was read.

This guide recommends three things that no source has published as practice:

  • the single definition of cost per resolved outcome, recorded in the decision log;
  • tokens per resolved outcome as the main meter;
  • one gateway key per registry entry.

No enterprise has published per-agent budgets running in production. The four meters have no published benchmark for a good value.

Next step: see how measured results feed back into the design on the learning flywheel, or return to Start here.

Explore this topic

Cite this page

Written by Murali Sid, Founder and AI Systems Architect, last updated 15 October 2026. Licensed CC BY-SA 4.0, so quote it freely with attribution.

Murali Sid (2026). Token efficiency: spend tokens on outcomes. Agentic Architecture Skills. https://www.agenticarchitectureskills.com/memory/token-efficiency

Machine-readable version of this page: /memory/token-efficiency.md

How context and memory work together

A larger reading window helps some tasks; it does not decide what to retain or trust.

When memory spans years and sites

Scale the search, updates, and evidence checks without losing the question.

On this page

Where the tokens goThe unit: cost per resolved outcomeWhat to meter for each agentTechniques with a measured effectBudgets tied to the registryWhat nobody has measuredThe incidents, correctedEvidence and limits