From Token Spend to Workflow Economics: What BCG’s Return-on-AI Framework Requires

A token bill can tell you what inference cost. It cannot tell you whether the work was accepted, whether a human had to redo it, or whether the business outcome arrived two weeks later. Managing AI economics requires a new unit of control: the workflow run, joined to its eventual outcome.

BCG makes a useful move: stop treating the token as the unit of management and start measuring the cost of a successful outcome. Its return-on-AI framing puts economic return over the combined cost of human intelligence and tokens. It also argues for a workflow-level operating model that can see spend, shape cost, and prove value—or stop the activity.

The idea is sound. The hard part is implementation.

Model gateways observe requests. Agent runtimes observe turns, tools, retries, and pauses. Finance sees invoices. Business systems observe whether a case closed, a payment settled, a pull request shipped, or an escalation reopened. None of those records, alone, is return on AI.

1. The token is a billing unit, not a business unit

Tokens remain essential telemetry. They explain input size, output size, cache behavior, and part of the cost of a model call. But they do not have stable business meaning across models or providers. Tokenizers differ. Input, output, cached input, cache creation, reasoning, audio, and other modalities can carry different billing rules. A million units in one product is not automatically equivalent to a million units in another.

Even within one pricing category, where billed cost may scale linearly with token volume, workflow economics need not. A larger context can change model behavior. A cheaper model can trigger more retries. A concise answer can increase human review. A cache hit can remove an inference call but return an answer that is stale for the user’s current state. An agent can enter a tool loop that spends little per call and a great deal per resolved case.

BCG identifies four interacting cost forces: adoption breadth and depth, task intensity, context and loops, and model mix. That framing matters because each force exists above the individual request. A single request is often too small to show the work; a monthly provider invoice is too large to assign responsibility or value.

2. Define two contracts before building the dashboard

A credible workflow-economics system begins with two explicit contracts: a cost envelope and an outcome contract. The first says which resources count. The second says what success means. If either remains implicit, cost-per-outcome becomes a number with an unstable denominator or a negotiable numerator.

The cost envelope

For each workflow run, capture the costs that materially change the decision. That usually includes model inference, tool and infrastructure usage, human oversight, and failure or rework. The categories should be mutually exclusive or governed by an allocation rule so the same correction time is not counted twice. The goal is not false precision. Human time, for example, may be sampled by workflow and role instead of metered for every click. The requirement is consistency: use the same inclusion rules when comparing policies.

The outcome contract

“Successful” must be a versioned, testable definition. For a support workflow, closure is not enough if the ticket reopens within seven days. For coding, lines generated are not enough if the change fails review or never ships. For document processing, a field extraction is not accepted until it passes domain validation or downstream reconciliation.

This is an operating measure, not a financial-reporting formula. When outcome value cannot be monetized responsibly, use cost per accepted outcome and report quality, latency, risk, and review load alongside it. Do not compress every objective into one dollar estimate just to create a ratio.

3. Build the attribution join

The cost and outcome contracts become useful only when their records can be joined. The join should be designed before optimization begins, because retrofitting ownership and outcome IDs after a cost spike usually leaves gaps.

A practical record carries at least:

• Workflow identity: workflow name and version, run ID, environment, owner, team, and cost center.
• Execution identity: agent, session, turn, request or trace ID, model policy, provider, tools, and prompt or skill versions.
• Cost evidence: inference, cache, tool, infrastructure, human-review, retry, and remediation cost, with the pricing source and version needed to reproduce the calculation.
• Outcome identity: outcome ID, source system, provisional or settled state, value method, and settlement timestamp.

Custom metadata is the connective tissue. A gateway request should carry the workflow run ID, owner, environment, customer or product boundary, and cost center where appropriate. The runtime should preserve the same run identity across turns and tool activity. The application should write that run ID beside the eventual outcome in the system of record.

Do not put sensitive content into tags merely because metadata is convenient. Use opaque identifiers, enforce an allowlist, and resolve business details inside the authorized analytics environment. Attribution should improve accountability without turning observability fields into a new data-leak surface.

4. Outcomes arrive later than costs

Inference cost is usually known within seconds. Business outcomes may settle minutes, days, or months later. A coding agent can finish its turn today while the change ships next week. A service conversation can close now and reopen later. A sales assistant can draft an email whose commercial effect is unknowable for a quarter.

The ledger therefore needs states:

• Open: the workflow is still running or waiting for a human.
• Provisional: execution ended, but the observation window is open.
• Accepted or rejected: the source of truth has evaluated the defined outcome.
• Reversed: a previously accepted outcome was undone, reopened, rolled back, or disputed.

This prevents a subtle reporting error: comparing today’s complete cost with today’s incomplete outcomes. Dashboards should show cohort windows and settlement coverage. If only 60% of runs are mature enough to judge, the apparent cost per success is provisional.

5. Separate correlation from causation

An outcome joined to an AI run is attributable in the database sense; it is not automatically caused by AI. Seasonality, staffing, policy changes, product changes, and selection effects can move the same metric.

For material decisions, establish a baseline and use an experimental or quasi-experimental design where feasible: randomized holdouts, staged rollouts, matched cohorts, or interrupted time-series analysis. At minimum, record the production policy version and compare like-for-like workflow segments.

This distinction is especially important when human effort moves rather than disappears. If an assistant drafts faster but creates more review work downstream, a local productivity metric can rise while end-to-end cost does not. The denominator must follow the workflow across organizational boundaries.

6. Optimize one lever at a time—and keep quality as a constraint

BCG proposes removing unnecessary model use, routing by task complexity, reusing context and components, and training for token discipline. Those are sensible levers. But each can move quality, latency, and risk, so optimization needs a guarded loop rather than a one-time cost cut.

Remove the model where the task is deterministic

Arithmetic, exact lookups, policy thresholds, schema validation, and deterministic routing often belong in conventional software. This can lower cost and improve reliability. The important design move is not to eliminate reasoning indiscriminately, but to preserve it for ambiguity and judgment.

Route by task complexity

A smaller or open-weight model may be the right workhorse for routine tasks, with a more capable model reserved for harder cases. But routing is an experiment, not a promise. Measure escalation rate, retries, accepted quality, latency, and review time by route. A lower request price that increases failure loops can make the outcome more expensive.

Reuse context deliberately

Stable prefixes, provider prompt caching, gateway response caching, batch execution, retrieval discipline, and lazy loading can reduce repeated work. They are different mechanisms with different semantics. Provider prompt caching reuses prompt processing while still generating a new response. Exact or semantic response caching can avoid the model call, but it needs isolation, expiry, and correctness controls. Versioned prompts and skills improve reuse and governance; they do not, by themselves, guarantee cacheability.

Bound loops and output

Set maximum turns, tool attempts, reasoning effort, context budgets, and output limits appropriate to the workflow. Record why a run stopped. A stopping rule should distinguish success, human handoff, policy block, timeout, exhausted budget, and unrecoverable error. Otherwise a cost reduction can simply appear as more abandoned work.

7. An illustrative comparison: cheaper requests, costlier outcomes

Consider a document-review workflow choosing between a lighter model and a more capable model. The figures below are normalized cost units for illustration, not benchmark data.

Policy A wins on inference cost and loses on the workflow. Policy B spends three times as much on the model call yet costs about 47% less per accepted outcome in this example. The point is not that larger models are always better. It is that model price cannot settle a workflow decision.

8. How the TrueFoundry and TrueForge stack maps to the problem

TrueFoundry provides several pieces of the denominator and the control loop. It does not manufacture the business numerator. That boundary is the key to a credible architecture.

That division also prevents three common category errors:

• A budget is a spend boundary, not evidence of value.
• A trace is execution evidence, not an evaluator.
• An agent’s final answer is not proof that an external side effect committed.

9. A 90-day implementation sequence

BCG frames the operating model as see, shape, and prove or stop. The following is our suggested technical starting sequence, not a BCG benchmark.

Days 1–30: make material spend attributable

1. Select five to ten production workflows with meaningful cost or risk.
2. Assign an owner, cost center, environment, and product-versus-internal classification.
3. Define a versioned accepted outcome and observation window for each workflow.
4. Propagate workflow and run IDs through model requests, runtime events, and the outcome system.
5. Reject or quarantine unattributed production traffic rather than labeling it “other” indefinitely.

Days 31–60: settle the ledger

1. Export gateway cost and trace data into the analytics environment.
2. Add TrueForge turn and event records for agent workflows.
3. Estimate human review and rework by workflow segment.
4. Join authoritative outcomes and distinguish provisional, accepted, rejected, and reversed states.
5. Publish cohort-aware cost-per-accepted-outcome dashboards with settlement coverage.

Days 61–90: change one lever and prove it

1. Rank workflows by total cost, cost per accepted outcome, and uncertainty in the measurement.
2. Pick one lever per experiment: model route, context policy, cache policy, tool path, output limit, or stopping rule.
3. Evaluate on a regression set, then canary in production with the same outcome contract.
4. Use budgets and alerts as safety rails while the new policy runs.
5. Scale, revise, or roll back based on settled outcomes—not early token savings.

10. The questions a useful dashboard should answer

A workflow-economics dashboard should help someone make a decision. At minimum:

• Which workflows have the highest total cost and the highest cost per accepted outcome?
• What percentage of spend has an owner, run ID, and defined outcome?
• Which policies default to frontier models, and do those routes improve accepted outcomes?
• How much input is served from provider prompt caches, and how much traffic is served from gateway response caches?
• Where do retries, tool loops, human review, and reversals dominate the cost envelope?
• How many outcomes are still provisional, and how does settlement change the result?
• Which apparent savings survive a controlled comparison against the baseline?

Provider spend still belongs on the dashboard. It simply should not be the last row.

11. What not to claim

12. The management question changes

The first useful question is not “How do we buy fewer tokens?” It is “Which workflow is spending, what accepted result is it producing, and what is the complete cost of that result?”

Once that join exists, token optimization becomes more precise. Teams can remove avoidable model calls, route routine work to appropriate models, cache safely, load only the context a task needs, and place budgets around experiments and production workloads. They can also discover the opposite: a more capable model, a longer context, or an extra verification step may increase inference cost and lower the cost of the accepted outcome.

That is the practical contribution of BCG’s return-on-AI idea. It moves the discussion from metered activity to workflow economics. The infrastructure consequence is equally clear: model telemetry, agent events, human work, and business outcomes need a shared identity and a disciplined settlement process.

Frequently asked questions

Is cost per token still useful?

Yes. It is useful for pricing validation, model-call analysis, and cost diagnostics within a comparable billing category. It is insufficient for deciding whether a workflow creates value.

What is the best unit for AI cost management?

For production applications, usually the workflow run: a bounded unit that contains model calls, tools, retries, human checkpoints, and a reference to an eventual outcome.

Does TrueFoundry calculate return on AI?

TrueFoundry supplies model cost, routing, cache, budget, identity, metadata, and trace evidence; TrueForge adds agent-runtime events and usage. The customer’s evaluators, systems of record, human-cost model, and analytics layer must supply and settle the business numerator.

Can a cheaper model increase total cost?

Yes. If it causes more retries, longer loops, escalations, or human correction, its cost per accepted outcome can exceed that of a more expensive model.

Should all AI costs be categorized as capex, opex, or COGS?

BCG uses those categories as a management framework. Actual accounting classification depends on applicable accounting standards, contracts, materiality, and organizational policy; finance and accounting owners should decide it.

References

1. BCG — Return on AI: How CFOs and CIOs Can Manage the Token Meter
2. TrueFoundry AI Gateway — Cost Tracking
3. TrueFoundry AI Gateway — Request Logs
4. TrueFoundry AI Gateway — Request Log Span Attributes
5. TrueFoundry AI Gateway — Metadata Validation
6. TrueFoundry AI Gateway — Virtual Model Routing
7. TrueFoundry AI Gateway — Exact and Semantic Caching
8. TrueFoundry AI Gateway — Budget Limiting
9. TrueFoundry AI Gateway — Prompt Management
10. TrueForge — Agent Events and Turn Metrics
11. TrueFoundry Agent Platform — Skills

Product documentation was validated on September 5, 2026. Capabilities and interfaces can change; verify current documentation before implementation.

‍