From Token Spend to Workflow Economics: What BCG’s Return-on-AI Framework Requires

A token bill can tell you what inference cost. It cannot tell you whether the work was accepted, whether a human had to redo it, or whether the business outcome arrived two weeks later. Managing AI economics requires a new unit of control: the workflow run, joined to its eventual outcome.

BCG makes a useful move: stop treating the token as the unit of management and start measuring the cost of a successful outcome. Its return-on-AI framing puts economic return over the combined cost of human intelligence and tokens. It also argues for a workflow-level operating model that can see spend, shape cost, and prove value—or stop the activity.

The idea is sound. The hard part is implementation.

Model gateways observe requests. Agent runtimes observe turns, tools, retries, and pauses. Finance sees invoices. Business systems observe whether a case closed, a payment settled, a pull request shipped, or an escalation reopened. None of those records, alone, is return on AI.

1. The token is a billing unit, not a business unit

Tokens remain essential telemetry. They explain input size, output size, cache behavior, and part of the cost of a model call. But they do not have stable business meaning across models or providers. Tokenizers differ. Input, output, cached input, cache creation, reasoning, audio, and other modalities can carry different billing rules. A million units in one product is not automatically equivalent to a million units in another.

Even within one pricing category, where billed cost may scale linearly with token volume, workflow economics need not. A larger context can change model behavior. A cheaper model can trigger more retries. A concise answer can increase human review. A cache hit can remove an inference call but return an answer that is stale for the user’s current state. An agent can enter a tool loop that spends little per call and a great deal per resolved case.

BCG identifies four interacting cost forces: adoption breadth and depth, task intensity, context and loops, and model mix. That framing matters because each force exists above the individual request. A single request is often too small to show the work; a monthly provider invoice is too large to assign responsibility or value.

2. Define two contracts before building the dashboard

A credible workflow-economics system begins with two explicit contracts: a cost envelope and an outcome contract. The first says which resources count. The second says what success means. If either remains implicit, cost-per-outcome becomes a number with an unstable denominator or a negotiable numerator.

The cost envelope

For each workflow run, capture the costs that materially change the decision. That usually includes model inference, tool and infrastructure usage, human oversight, and failure or rework. The categories should be mutually exclusive or governed by an allocation rule so the same correction time is not counted twice. The goal is not false precision. Human time, for example, may be sampled by workflow and role instead of metered for every click. The requirement is consistency: use the same inclusion rules when comparing policies.

The outcome contract

“Successful” must be a versioned, testable definition. For a support workflow, closure is not enough if the ticket reopens within seven days. For coding, lines generated are not enough if the change fails review or never ships. For document processing, a field extraction is not accepted until it passes domain validation or downstream reconciliation.

This is an operating measure, not a financial-reporting formula. When outcome value cannot be monetized responsibly, use cost per accepted outcome and report quality, latency, risk, and review load alongside it. Do not compress every objective into one dollar estimate just to create a ratio.

3. Build the attribution join

The cost and outcome contracts become useful only when their records can be joined. The join should be designed before optimization begins, because retrofitting ownership and outcome IDs after a cost spike usually leaves gaps.

A practical record carries at least:

• Workflow identity: workflow name and version, run ID, environment, owner, team, and cost center.
• Execution identity: agent, session, turn, request or trace ID, model policy, provider, tools, and prompt or skill versions.
• Cost evidence: inference, cache, tool, infrastructure, human-review, retry, and remediation cost, with the pricing source and version needed to reproduce the calculation.
• Outcome identity: outcome ID, source system, provisional or settled state, value method, and settlement timestamp.

Custom metadata is the connective tissue. A gateway request should carry the workflow run ID, owner, environment, customer or product boundary, and cost center where appropriate. The runtime should preserve the same run identity across turns and tool activity. The application should write that run ID beside the eventual outcome in the system of record.

Do not put sensitive content into tags merely because metadata is convenient. Use opaque identifiers, enforce an allowlist, and resolve business details inside the authorized analytics environment. Attribution should improve accountability without turning observability fields into a new data-leak surface.

4. Outcomes arrive later than costs

Inference cost is usually known within seconds. Business outcomes may settle minutes, days, or months later. A coding agent can finish its turn today while the change ships next week. A service conversation can close now and reopen later. A sales assistant can draft an email whose commercial effect is unknowable for a quarter.

The ledger therefore needs states:

• Open: the workflow is still running or waiting for a human.
• Provisional: execution ended, but the observation window is open.
• Accepted or rejected: the source of truth has evaluated the defined outcome.
• Reversed: a previously accepted outcome was undone, reopened, rolled back, or disputed.

This prevents a subtle reporting error: comparing today’s complete cost with today’s incomplete outcomes. Dashboards should show cohort windows and settlement coverage. If only 60% of runs are mature enough to judge, the apparent cost per success is provisional.

5. Separate correlation from causation

An outcome joined to an AI run is attributable in the database sense; it is not automatically caused by AI. Seasonality, staffing, policy changes, product changes, and selection effects can move the same metric.

For material decisions, establish a baseline and use an experimental or quasi-experimental design where feasible: randomized holdouts, staged rollouts, matched cohorts, or interrupted time-series analysis. At minimum, record the production policy version and compare like-for-like workflow segments.

This distinction is especially important when human effort moves rather than disappears. If an assistant drafts faster but creates more review work downstream, a local productivity metric can rise while end-to-end cost does not. The denominator must follow the workflow across organizational boundaries.

6. Optimize one lever at a time—and keep quality as a constraint

BCG proposes removing unnecessary model use, routing by task complexity, reusing context and components, and training for token discipline. Those are sensible levers. But each can move quality, latency, and risk, so optimization needs a guarded loop rather than a one-time cost cut.

Remove the model where the task is deterministic

Arithmetic, exact lookups, policy thresholds, schema validation, and deterministic routing often belong in conventional software. This can lower cost and improve reliability. The important design move is not to eliminate reasoning indiscriminately, but to preserve it for ambiguity and judgment.

Route by task complexity

A smaller or open-weight model may be the right workhorse for routine tasks, with a more capable model reserved for harder cases. But routing is an experiment, not a promise. Measure escalation rate, retries, accepted quality, latency, and review time by route. A lower request price that increases failure loops can make the outcome more expensive.

Reuse context deliberately

Stable prefixes, provider prompt caching, gateway response caching, batch execution, retrieval discipline, and lazy loading can reduce repeated work. They are different mechanisms with different semantics. Provider prompt caching reuses prompt processing while still generating a new response. Exact or semantic response caching can avoid the model call, but it needs isolation, expiry, and correctness controls. Versioned prompts and skills improve reuse and governance; they do not, by themselves, guarantee cacheability.

Bound loops and output

Set maximum turns, tool attempts, reasoning effort, context budgets, and output limits appropriate to the workflow. Record why a run stopped. A stopping rule should distinguish success, human handoff, policy block, timeout, exhausted budget, and unrecoverable error. Otherwise a cost reduction can simply appear as more abandoned work.

7. An illustrative comparison: cheaper requests, costlier outcomes

Consider a document-review workflow choosing between a lighter model and a more capable model. The figures below are normalized cost units for illustration, not benchmark data.

‍