Budget Limiting - TrueFoundry Docs

Budget limiting helps you control spending on LLM workloads by setting cost boundaries per tenant or team. You define budget rules that match specific users, models, or metadata, and the gateway blocks requests that would exceed a limit — or runs in warn-only mode to monitor spending without blocking.

How budget limiting works

Budget limiting consists of a set of independent rules. Each rule defines which requests it applies to and how much they can spend. During evaluation:

Every matching rule is checked. The gateway finds all rules whose scope and filters match the incoming request.
All matching rules must allow the request. If any matching rule has exceeded its budget and is in enforcement mode, the request is blocked.
Cost is tracked against every matching rule. When a request matches multiple rules, usage increments on each of them.

Think of matching rules as an AND across budgets: a request proceeds only when every rule it touches is within its limit.

Budget Limiting rules list showing tenant and team scoped rules

The rules page summarizes this behavior: every matching rule is evaluated, and the request is blocked if any rule it touches is breached.

Budget Limiting overview with Add Rule button

Tenant and team scopes

Budget rules are created at one of two scopes:

Scope	Who can manage	Applies to
Tenant budget	Tenant admins only	All requests across the tenant that match the rule’s filters
Team budget	Tenant admins and team managers	Requests from members of a single team that match the rule’s filters

Use tenant budgets for organization-wide caps (for example, a monthly GPT-4 spend limit). Use team budgets when a team lead should manage their own team’s spending without tenant-admin access.

To create a rule, go to AI Gateway → Policies → Budget Limiting and click Add Rule. Select Tenant Budget or Team Budget as the scope, then configure the rule.

Setting up a budget rule

The rule form has two steps: Select a scope and Configure.

Scope filters

Define which requests the rule applies to. Filters use AND logic — a request must match every filter you add.

Filter	Operators	Description
Subjects	`IN`, `NOT IN`	Users, teams, or virtual accounts
Models	`IN`, `NOT IN`	Model names served through the gateway
Metadata	`IN`, `NOT IN`	Key-value pairs from the `X-TFY-METADATA` header

Use + Add Filters to combine subjects, models, and metadata on the same rule.

Budget rule scope filters with Subjects, Models, and Metadata

If you leave all filters empty, the rule matches every request within its tenant or team scope. This is useful for default budgets that apply broadly.

Budget limits

Set one or more spending limits on the same rule using + Add Period. Each limit has an amount and a reset period:

Reset period	Behavior
Daily	Resets daily at 00:00 UTC
Weekly	Resets every Monday at 00:00 UTC
Monthly	Resets on the 1st of every month at 00:00 UTC
Quarterly	Resets on January 1, April 1, July 1, and October 1 at 00:00 UTC
Lifetime	Never resets — usage accumulates for the life of the rule

Budget rule form with reset period options including Lifetime

A single rule can enforce multiple periods at once. For example, you can set a $0.50/day cap and a $2/quarter cap on the same rule. The request is blocked if any period on that rule is exceeded.

Budget tracking starts from rule creation, not from the beginning of the period.When you create a budget rule, the usage counter starts at $0 from that moment — regardless of how much was spent earlier in the current day, week, month, or quarter. Prior spending is not retroactively counted.For lifetime budgets, usage accumulates from the rule’s creation date forward and never resets.

Apply limit as

Controls how the budget is partitioned across matching requests:

Value	Effect
Aggregate (shared)	One shared budget pool for all matching requests
Per user	Each user gets an independent budget
Per model	Each model gets an independent budget
Per virtual account	Each virtual account gets an independent budget
Per metadata	Each unique metadata key value gets an independent budget

Overrides

When you choose any per-entity option (per user, per model, per virtual account, or per metadata), you can set overrides — per-entity exceptions that replace the base limits for specific entities. Every other matching entity keeps the base limits; only the listed entities use their override values.

Use + Add Override, select the entity, and set a custom amount for each period. An override replaces all periods for that entity, so include every period you want to apply. For example, on a per-user rule with a base limit of $0.01/day and $0.10/month, you can give one user their own $1/day and $5/month allowance while everyone else stays on the base limits.

Overrides are only available for per-entity rules. Aggregate (shared) rules use a single pool, so there are no individual entities to override.

Enforcement mode

Block If Usage Limit Exceeded controls whether the rule enforces the budget or runs in warn-only mode:

ON (enforcement mode): Requests are rejected with a budget-exceeded error once usage crosses the limit. In YAML, this is mode: enforce.
OFF (warn-only mode): Requests are allowed through even when the budget has been exceeded. Usage is still tracked and alerts still fire.

Warn-only mode is useful for validating a new budget before turning on enforcement, or for visibility without hard cutoffs on critical paths.

Budget milestone alerts

Enable Send Alerts On Budget Milestones to notify your team when usage crosses percentage thresholds. Select thresholds (75%, 90%, 95%, 100%) and choose a notification channel (email, Slack webhook, or Slack bot).

Alert configuration details

Available thresholds: 75%, 90%, 95%, 100%Each threshold triggers once per budget period. When a new period starts (or for lifetime budgets, once per threshold crossing), alerts can fire again. Alerts are checked every 20 minutes.Notification channels:

Email — Send alerts to one or more email addresses via a configured email notification channel
Slack Webhook — Send alerts to a Slack channel via a webhook notification channel
Slack Bot — Send alerts to specific Slack channels via a bot notification channel

Viewing budget usage

Each rule card on the Budget Limiting page shows:

Current usage amount and percentage per limit period
Budget limit and remaining budget
Period start time (when the current budget period began)
Active use breakdown for per-user, per-model, or per-metadata rules

For rules with per-entity budgets, you can see how many entities are within budget and how many are over.

Practical examples

Each example shows both the UI configuration and the equivalent YAML. Click Apply as YAML in the rule form to paste or edit YAML directly.

Model spend cap for specific users

Cap monthly spending on a specific model for a set of users. Every matching request must stay under the limit.

UI
YAML

Scope	Rule name	Filters	Budget	Apply as
Tenant	`model-spend-cap`	Model: `tfy-ai-databricks/databricks-claude-sonnet-4-6`, User: `vinit.kanse@truefoundry.com`	$1/month	Aggregate (shared)

How it works: Any request from the specified user to the specified model counts against a single shared $1/month pool. If usage exceeds $1, matching requests are blocked.

name: model-spend-cap
type: tenant-budget-config
mode: enforce
applies_to:
  type: aggregate
limits:
  cost_per_month: 1
when:
  subjects:
    users:
      in:
        - vinit.kanse@truefoundry.com
  models:
    in:
      - tfy-ai-databricks/databricks-claude-sonnet-4-6

Per-user daily limits with a team budget

A team manager sets a $10/day per-user limit for their team without affecting other teams.

UI
YAML

Scope	Rule name	Filters	Budget	Apply as
Team (`fe-team-mini`)	`per-user-daily`	(no filters — all team requests)	$10/day	Per user

How it works: Each member of fe-team-mini gets an independent $10/day budget. Any tenant-wide rule that also matches is evaluated separately — both must pass.

name: per-user-daily
type: team-budget-config
team: fe-team-mini
mode: enforce
applies_to:
  type: user
limits:
  cost_per_day: 10
when: {}

Per-user budget with overrides

Set a low per-user base limit, then raise it for specific users with overrides.

UI
YAML

Scope	Rule name	Filters	Budget	Apply as
Team (`fe-team-mini`)	`per-user-with-overrides`	Model: `databricks-claude-opus-4-5`	$0.01/day, $0.10/month	Per user

Overrides: abhishek.choudhary@truefoundry.com → $1/day, $5/month. Every other user stays on the base $0.01/day and $0.10/month limits.

name: per-user-with-overrides
type: team-budget-config
team: fe-team-mini
mode: enforce
applies_to:
  type: user
limits:
  cost_per_day: 0.01
  cost_per_month: 0.1
when:
  models:
    in:
      - tfy-ai-databricks/databricks-claude-opus-4-5
overrides:
  - subject: user:abhishek.choudhary@truefoundry.com
    limits:
      cost_per_day: 1
      cost_per_month: 5

Multi-period limits on one rule

Enforce daily, monthly, and quarterly caps in a single rule.

UI
YAML

Scope	Rule name	Budget	Apply as
Tenant	`multi-period-cap`	$0.50/day, $0.50/month, $2/quarter	Aggregate (shared)

How it works: Usage is tracked against all three periods simultaneously. A request is blocked if any period on this rule is exceeded.

name: multi-period-cap
type: tenant-budget-config
mode: enforce
applies_to:
  type: aggregate
limits:
  cost_per_day: 0.5
  cost_per_month: 0.5
  cost_per_quarter: 2
when: {}
alerts:
  thresholds: [90, 100]
  send_to: shared
  notification_target:
    - type: email
      notification_channel: budget-alerts-channel
      to_emails:
        - admin@example.com

Lifetime budget with warn-only mode

Set a spend cap that never resets — useful for pilot programs or one-time allocations.

UI
YAML

Scope	Rule name	Filters	Budget	Apply as
Tenant	`pilot-lifetime-cap`	Team: `pilot-team`	$1,000/lifetime	Aggregate (shared)

How it works: Usage accumulates from rule creation and never resets. With Block If Usage Limit Exceeded off, requests are never blocked — alerts still fire at each threshold so you can monitor the pilot’s spend.

name: pilot-lifetime-cap
type: tenant-budget-config
mode: audit
applies_to:
  type: aggregate
limits:
  cost_lifetime: 1000
when:
  subjects:
    teams:
      in:
        - pilot-team
alerts:
  thresholds: [75, 90, 95, 100]
  send_to: shared
  notification_target:
    - type: slack-webhook
      notification_channel: pilot-alerts-channel

Metadata-based project budgets

Track spending per project using metadata sent in the X-TFY-METADATA header.

UI
YAML

Scope	Rule name	Filters	Budget	Apply as
Tenant	`project-daily-budget`	Metadata: `environment IN production`	$100/day	Per metadata (`project_id`)

Requests must include the header:

X-TFY-METADATA: {"project_id": "proj-123", "environment": "production"}

Each unique project_id value gets its own $100/day budget.

name: project-daily-budget
type: tenant-budget-config
mode: enforce
applies_to:
  type: metadata
  key: project_id
limits:
  cost_per_day: 100
when:
  metadata:
    environment:
      in:
        - production

Independent safety nets

Combine a per-user default with a model-wide cap. Both rules apply to every matching request.

UI
YAML

Scope	Rule name	Filters	Budget	Apply as
Tenant	`per-user-daily`	(no filters)	$10/day	Per user
Tenant	`gpt4-monthly-cap`	Model: `openai-main/gpt-4`	$500/month	Aggregate (shared)

How it works: A user calling GPT-4 is checked against both rules. The per-user $10/day limit and the $500/month model-wide cap must both be satisfied for the request to proceed.

# Rule 1 — per-user daily default
name: per-user-daily
type: tenant-budget-config
mode: enforce
applies_to:
  type: user
limits:
  cost_per_day: 10
when: {}
---
# Rule 2 — model-wide monthly cap
name: gpt4-monthly-cap
type: tenant-budget-config
mode: enforce
applies_to:
  type: aggregate
limits:
  cost_per_month: 500
when:
  models:
    in:
      - openai-main/gpt-4

YAML configuration reference

YAML structure reference

name: model-spend-cap
type: tenant-budget-config
mode: enforce
applies_to:
  type: aggregate
limits:
  cost_per_month: 1
when:
  subjects:
    users:
      in:
        - vinit.kanse@truefoundry.com
  models:
    in:
      - tfy-ai-databricks/databricks-claude-sonnet-4-6
  metadata:
    key:
      in:
        - value
alerts:
  thresholds:
    - 75
    - 90
    - 95
    - 100
  send_to: shared
  notification_target:
    - type: slack-webhook
      notification_channel: truefoundry:slack:test-slack:notification-channel:testing

Field reference:

Field	Description
`name`	Unique rule identifier
`type`	`tenant-budget-config` for tenant-scoped rules, `team-budget-config` for team-scoped rules
`mode`	`enforce` (block when exceeded) or `audit` (warn-only — track and alert but don’t block)
`applies_to.type`	`aggregate`, `user`, `model`, `virtualaccount`, or `metadata` (with `key` for metadata)
`limits`	One or more limit keys: `cost_per_day`, `cost_per_week`, `cost_per_month`, `cost_per_quarter`, `cost_lifetime`
`overrides`	Optional. Per-entity exceptions (per-entity rules only). Each entry names an entity and its own `limits` block, which replaces the base limits for that entity
`when.subjects`	Filter by users, teams, or virtual accounts using `in` / `not_in`
`when.models`	Filter by model names using `in` / `not_in`
`when.metadata`	Filter by metadata keys using `in` / `not_in`
`alerts.thresholds`	Percentage thresholds: `75`, `90`, `95`, `100`
`alerts.send_to`	`shared` for aggregate alerts, or per-entity routing when `applies_to` is not aggregate
`alerts.notification_target`	Notification channel configuration (email, `slack-webhook`, or `slack-bot`)

​How budget limiting works

​Tenant and team scopes

​Setting up a budget rule

​Scope filters

​Budget limits

​Apply limit as

​Overrides

​Enforcement mode

​Budget milestone alerts

​Viewing budget usage

​Practical examples

​YAML configuration reference

How budget limiting works

Tenant and team scopes

Setting up a budget rule

Scope filters

Budget limits

Apply limit as

Overrides

Enforcement mode

Budget milestone alerts

Viewing budget usage

Practical examples

YAML configuration reference