> ## Documentation Index
> Fetch the complete documentation index at: https://www.truefoundry.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Cost Savings

> See how the Metrics Overview reports savings from prompt caching, semantic caching, and auto routing, and how each figure is calculated.

The AI Gateway estimates how much you saved versus full-price inference. The Metrics Overview reports that total, then splits it into three independent sources: provider prompt caching, gateway semantic/exact caching, and Auto Routing.

## Where cost savings appear

Open **AI Gateway** → **AI Monitoring** → **Metrics** → **Overview**.

The usage table includes a **Cost savings** column next to **Spend**. Each row shows the dollar amount saved for that user, virtual account, or team, plus a stacked bar of the three sources. Hover the cell for the breakdown, the total saved, and the percent off full price.

<Note>
  The [Cache Metrics](/docs/ai-gateway/analytics#cache-metrics) tab also has a **Total Cost Saved** counter. That counter is gateway cache hits only (semantic and exact-match). The Overview **Cost savings** figure is the sum of all three sources below.
</Note>

## What the breakdown shows

| Source | Label in the UI | What it measures |
| - | - | - |
| Prompt caching | **Prompt caching** | Cached input tokens billed at the provider's discount instead of the full input price |
| Semantic / exact caching | **Semantic caching** | Requests answered from the gateway cache with no model call |
| Auto Routing | **Auto-routing** | Requests routed to a cheaper model that still met the quality bar |

The percent shown next to the total is `saved / (spend + saved)` — the share of what the same traffic would have cost at full price.

## How each figure is calculated

In every formula below, **A** is the actual cost of the request: today's `tfy.model.metric.cost_in_usd`.

### Prompt caching

Provider prompt caching reuses a previously computed prompt prefix. Cached input tokens are billed at a lower rate than uncached input. See [Prompt caching](/docs/ai-gateway/chat-completions-advanced#prompt-caching).

Savings are:

```text theme={"dark"}
prompt cache savings = (same request billed as if every input token were uncached) − A
```

The gateway only records this when the request actually read from or wrote to the provider prompt cache. The value can be negative on a request that mostly **writes** to the cache, because cache writes are often priced above the normal input rate.

### Semantic caching

Gateway [exact-match and semantic caching](/docs/ai-gateway/caching) return a stored response without calling the model. On a hit, **A** is `$0`.

Savings are the cost of the original request that populated the cache — the amount you would have paid if that hit had been a miss. The same number is stored as `tfy.model.cache.potential_cost_savings_in_usd` and aggregated as `potentialCostSavings` in [Cache Metrics](/docs/ai-gateway/fetch-cache-metrics).

This bucket includes both exact-match and semantic hits. The Overview labels it **Semantic caching** because semantic cache is a superset of exact-match.

### Auto Routing

[Auto Routing](/docs/ai-gateway/auto-routing) classifies each request and serves it with the cheapest configured model that can handle that complexity. Savings are estimated against the **baseline** model: the `complex` tier target — the most capable, and typically most expensive, model in the rule.

```text theme={"dark"}
auto routing savings = (the same tokens priced on the baseline model) − A
```

A few details:

* The figure is **0** when the request was already served by the baseline.
* It is an **estimate**: the baseline model might have used a different number of tokens for the same prompt.
* It is omitted when **A** is `$0` (for example a gateway cache hit), so a cached Auto Routing request is not counted twice.
* It is omitted when the baseline model's pricing cannot be resolved.

## How the total is combined

```text theme={"dark"}
total savings = prompt caching + semantic/exact caching + auto routing
```

The three numbers are computed independently per request and summed in the Overview. A single request can contribute more than one source — for example Auto Routing plus prompt caching on the cheaper model — but a gateway cache hit does not also claim Auto Routing savings.

<Tip>
  Cost savings appear only after the underlying feature is enabled and traffic uses it. Set up [prompt caching](/docs/ai-gateway/chat-completions-advanced#prompt-caching), [gateway caching](/docs/ai-gateway/caching), or [Auto Routing](/docs/ai-gateway/auto-routing) first, then return to Overview to see the breakdown.
</Tip>

## Related

<Columns cols={3}>
  <Card title="Cost tracking" href="/docs/ai-gateway/cost-tracking">
    How actual cost is priced from public or private rates.
  </Card>

  <Card title="Metrics dashboard" href="/docs/ai-gateway/analytics">
    Overview, cache metrics, and the rest of the dashboard.
  </Card>

  <Card title="Request log attributes" href="/docs/ai-gateway/fetch-request-logs-span-attributes">
    Per-request savings fields on model spans.
  </Card>
</Columns>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.