What BYOK means on an AI Gateway
By Ashish Dubey · Published October 8, 2026 · Updated October 15, 2026
Store each provider key once in a TrueFoundry model account; applications call the gateway with a TrueFoundry Personal Access Token or Virtual Account Token and never see the provider key. Access is set per model account with Model Account Manager and Model Account User roles, and keys rotate centrally without touching application code.
Virtual Account Tokens are tied to a virtual identity and are recommended for production applications, so each service ships with its own VAT instead of a shared provider key.
BYOK, or bring your own key, means teams keep their own accounts with model providers like OpenAI, Anthropic, and AWS Bedrock, and they plug those provider keys into the gateway once. Their applications never touch the raw provider keys again. They talk to one TrueFoundry AI Gateway endpoint using a single gateway key, and the gateway forwards each call to the right provider with their credentials.
This is the opposite of a reseller model where teams buy tokens from a middleman. With BYOK teams keep their own provider contracts, pricing, and rate limits, and they add centralized governance on top.
Why route provider keys through the gateway
Handing raw provider keys to every application and notebook is how key sprawl starts. Keys leak into code, get copied into environments, and become impossible to rotate without breaking something.
- One key for callers. Applications authenticate with a TrueFoundry key, not the provider key. As the docs put it, to access models through the gateway teams use TrueFoundry API keys, not the original provider keys.
- Central rotation. Provider keys live in one place. Rotate them in the gateway and every application keeps working.
- Access control. Teams decide who can use which model account, with manager and user roles per account.
- One interface. Instead of managing separate SDKs, endpoints, and keys for OpenAI, Anthropic, Bedrock, and self-hosted models, applications talk to one gateway endpoint and use one gateway key.
Adding a team's provider keys: model accounts
In TrueFoundry, a provider key lives inside a model account. The quick start docs describe a model account as one account of a model provider, for example OpenAI, Anthropic, or AWS Bedrock. Teams can add multiple accounts per provider, each with their own API keys, and each account can have multiple models.
To add one, select the provider the team wants, then add models after providing the API key. Teams can add multiple keys from the same provider by creating separate model accounts, which is useful when teams want to separate spend or rate limits by environment or team.
Once submitted, the team's model accounts and models appear under the Models tab. To add more later, go to AI Gateway, then Models, select a model account on the left, and click Add Model in the top right.
How callers authenticate: virtual keys
Once a team's provider keys are stored, applications never see them. The gateway access control docs describe two token types that callers use instead.
- Personal Access Tokens (PATs) are tied to a user and are recommended for developers during development.
- Virtual Account Tokens (VATs) are tied to a virtual identity and are recommended for production applications. This is the virtual key that a team's services ship with.
Access is configured at the model account level with two roles. A Model Account Manager can modify settings, add or remove models, and manage access permissions. A Model Account User can use all models within the account but cannot change settings or permissions. Tenant admins automatically have access to all models across the platform.
To call the gateway, teams need three things from the Code Snippet tab of the Playground: the gateway base URL, an API key, and a model id. SaaS users point to the gateway base URL, authenticate with a PAT or VAT, and use the standard OpenAI client.
from openai import OpenAI
client = OpenAI(
api_key="your_truefoundry_api_key",
base_url="https://gateway.truefoundry.ai",
)
What teams get once keys live in the gateway
- A single OpenAI compatible endpoint for every provider and every self-hosted model.
- Central key rotation without touching application code.
- Per account access control with manager and user roles.
- Centralized observability and policy, since every call flows through one place.
As one customer put it, the gateway eliminated the overhead of managing keys, routing logic, and scattered observability, and introducing new models became just configuration.