How Much Inference a Workflow Deserves

The next serious fight over AI budgets will look dull on the surface. It will start with a line item for token consumption.

For two years many executives treated tokens as a technical detail, buried in a vendor bill or smoothed over by a flat license. Inference turns AI into an operating cost. Training creates capability and inference consumes it, so every prompt, tool call, retrieved document, planning loop and agent retry converts strategy into spend.

Operators should therefore pay attention to the infrastructure debate. McKinsey's June 2026 analysis shows cost per token and energy per token becoming central measures of AI economics. Goldman Sachs Research projects that agentic AI will drive a 24-fold rise in token consumption by 2030, and Goldman Sachs Asset Management notes that enterprise token use is already diverging between heavy adopters and median firms. Meanwhile the National Institute of Standards and Technology (NIST) and the Center for AI Standards and Innovation are advancing agent standards and identity work, because agents need access to real systems before they produce real value.

Agents that do real work raise token volume, and higher volume makes cost and energy per token worth managing closely. Once those agents touch real systems, identity and authorization become budget questions too. Engineering can no longer own the budget alone.

Once agents do the work, the AI budget stops being an engineering line item.

AI used to be budgeted like software

Old AI budgets were mostly procurement questions: which model or copilot, how many seats, what vendor contract. Spend looked like software, so it was governed like software.

That held while usage stayed light, experimental or limited by how often people typed prompts. A team could buy a tool, run pilots and accept a loose link between activity and value, and the main question was whether anyone used it.

Agentic systems change the cost structure. A person requests an outcome, and behind that request the system may plan, call tools, query another model and retry a failed path, so a single answer on screen can sit on top of a long chain of inference events.

That gap carries managerial risk. One employee request leaves a trail across the company: the system consumes budget, the vendor records usage, finance gets a bill and security sees a new actor touching its systems. None of those parties decides whether the work was worth doing; someone in the business still has to.

Pricing tokens means pricing work

Seat pricing taught managers to think in users. Token pricing forces them to think in units of work.

A user can be cheap and wasteful or expensive and valuable. Tokens might buy fraud triage or a useful code review, or they might buy a hallucinated summary, a low-value slide rewrite, or an agent loop that should have stopped five calls earlier. What matters is the judgment or action the token pays for, and what that is worth.

I think blanket AI rationing is a crude answer. Forcing every workflow onto a small model can erase the economic value that justified using AI in the first place, while running frontier inference on work that a smaller model, a script or a human decision would handle wastes money in the other direction.

Five lines in a token budget

Executives need a clear picture of where tokens go before they need a dashboard. A useful token budget has five lines, set per workflow: model tier, context budget, agent loop, value class and owner.

Falling token prices make budgeting more necessary

McKinsey identifies model optimization, advanced packaging, custom silicon and co-packaged optics as major levers for reducing inference cost over time. If those improvements arrive, more workflows become economically plausible, and the familiar pattern returns: when the unit gets cheaper, usage expands faster than control over it.

Cheap tokens can be wasted as easily as expensive ones, and an agent running on them can touch the wrong system just the same. A cheap summary can mislead a decision, retrieval can expose sensitive context, and more machine output means more for humans to review.

Cost decline should widen the set of workflows worth testing without removing the judgment about which ones to run. As tokens get cheaper, the question moves from whether a workflow can run to whether it should consume intelligence automatically.

Agents spend authority as well as tokens

The token budget also needs a security line, because agents act on systems in addition to answering questions.

The NIST AI Agent Standards Initiative frames adoption around trust, interoperability, security and identity. Its Request for Information summary says respondents widely agreed that AI agents create novel security threats and that those concerns block adoption, which puts security in the budget discussion rather than in an appendix.

Once an agent has permissions, spending tokens can mean spending authority. It may read a customer record, update a Customer Relationship Management field or query a privileged database, and each step carries both an inference cost and an authorization cost, so the two should be optimized together.

Without controls, the company has handed a probabilistic worker a company card. For serious workflows, pair the token budget with identity controls: a named human owner, scoped permissions, just-enough access, an audit trail, a step limit, an exception path and a rollback rule.

Three rules for founders

A founder does not need an enterprise financial operations (FinOps) team to start. Three rules are enough.

1. Put every AI workflow into a value class before scaling it: revenue, delivery, risk, learning or convenience.
2. Set a maximum loop depth for agentic work, counted in tool calls, retries, documents or minutes before human review.
3. Hold one weekly token spend review covering top workflows by spend, output produced, human review burden and stop-rule exceptions.

These rules guard against two common founder mistakes: starving a valuable workflow because the bill feels new, and subsidizing an impressive one because the output feels magical. A workflow that cannot name the business metric its token spend improves stays experimental.

What the CFO should ask for

A single AI cost number hides the decision, and the CFO should refuse to accept one.

Separate token spend into strategic work, operating throughput, employee convenience and waste. Coding assistants that reduce review bottlenecks may deserve more expensive inference than presentation helpers. Fraud agents may justify heavy context retrieval if they reduce loss or escalation. Meeting-summary tools often need a hard ceiling, because the upside is modest and the volume is large.

The CFO's job is to price autonomy correctly rather than to suppress consumption, so the budget should reward high-value inference and force low-value workflows to simplify, move to a cheaper model, cache, use a template or stop.

Six questions before approving a large AI workflow

Ask for written answers to each of these before sign-off, so that approving more AI does not mean approving unmanaged inference.

1. The default model tier, and the event that allows escalation.
2. Which context may enter the prompt or retrieval path.
3. The maximum number of agent steps before a human owns the decision.
4. The business value class that justifies the token spend.
5. The human who owns the bill, the permissions and the rollback.
6. The weekly report that would expand, downshift or kill the workflow.

Both sides of the cost argument can be right

The market argument will keep swinging between abundance and scarcity. One side will point to falling cost per token; the other will point to infrastructure strain, energy limits, chip shortages and rising agent usage. Both can be right at once.

The operator's position is narrower. AI will be cheap enough to enter many workflows and expensive enough to punish lazy design, which favors companies that allocate tokens the way they allocate capital, with a stated expected return for each workflow and reporting that shows whether the return arrived.