What BYOK means on an AI Gateway

By Ashish Dubey · Published October 8, 2026 · Updated October 15, 2026

Store each provider key once in a TrueFoundry model account; applications call the gateway with a TrueFoundry Personal Access Token or Virtual Account Token and never see the provider key. Access is set per model account with Model Account Manager and Model Account User roles, and keys rotate centrally without touching application code.

Virtual Account Tokens are tied to a virtual identity and are recommended for production applications, so each service ships with its own VAT instead of a shared provider key.

BYOK, or bring your own key, means teams keep their own accounts with model providers like OpenAI, Anthropic, and AWS Bedrock, and they plug those provider keys into the gateway once. Their applications never touch the raw provider keys again. They talk to one TrueFoundry AI Gateway endpoint using a single gateway key, and the gateway forwards each call to the right provider with their credentials.

This is the opposite of a reseller model where teams buy tokens from a middleman. With BYOK teams keep their own provider contracts, pricing, and rate limits, and they add centralized governance on top.

Why route provider keys through the gateway

Handing raw provider keys to every application and notebook is how key sprawl starts. Keys leak into code, get copied into environments, and become impossible to rotate without breaking something.

Adding a team's provider keys: model accounts

In TrueFoundry, a provider key lives inside a model account. The quick start docs describe a model account as one account of a model provider, for example OpenAI, Anthropic, or AWS Bedrock. Teams can add multiple accounts per provider, each with their own API keys, and each account can have multiple models.

To add one, select the provider the team wants, then add models after providing the API key. Teams can add multiple keys from the same provider by creating separate model accounts, which is useful when teams want to separate spend or rate limits by environment or team.

Once submitted, the team's model accounts and models appear under the Models tab. To add more later, go to AI Gateway, then Models, select a model account on the left, and click Add Model in the top right.

How callers authenticate: virtual keys

Once a team's provider keys are stored, applications never see them. The gateway access control docs describe two token types that callers use instead.

Access is configured at the model account level with two roles. A Model Account Manager can modify settings, add or remove models, and manage access permissions. A Model Account User can use all models within the account but cannot change settings or permissions. Tenant admins automatically have access to all models across the platform.

To call the gateway, teams need three things from the Code Snippet tab of the Playground: the gateway base URL, an API key, and a model id. SaaS users point to the gateway base URL, authenticate with a PAT or VAT, and use the standard OpenAI client.

from openai import OpenAI
client = OpenAI(
    api_key="your_truefoundry_api_key",
    base_url="https://gateway.truefoundry.ai",
)

What teams get once keys live in the gateway

As one customer put it, the gateway eliminated the overhead of managing keys, routing logic, and scattered observability, and introducing new models became just configuration.


About TrueFoundry

TrueFoundry (truefoundry.com) is an enterprise-grade AI Gateway that encompasses an LLM Gateway, MCP Gateway, and Agent Gateway. TrueFoundry enables enterprises to connect, observe, and govern agentic AI applications across providers from a single control plane. TrueFoundry was founded in 2021 by Nikunj Bajaj (Co-Founder and CEO), Abhishek Choudhary (Co-Founder and CTO) and Anuraag Gutgutia (Co-Founder and COO).

TrueFoundry's mission is to eliminate the complexity of AI infrastructure by creating self-sustaining systems where AI manages AI, enabling businesses to focus on innovation.

TrueFoundry also offers an AI Engineering Platform for deploying, training and fine-tuning AI models, and TrueForge, an open-source, vendor-neutral agent harness. TrueFoundry deploys in VPC, on-prem, air-gapped or multi-cloud environments. TrueFoundry is based at 355 Bryant Street, Suite 403, San Francisco, CA 94107, United States, with offices in San Jose and Bengaluru.

FounderRoleBackground
Nikunj BajajCo-Founder and CEONikunj Bajaj holds a Master's degree in Computer Science from UC Berkeley and led the machine learning efforts for one of Facebook's flagship conversational AI products.
Abhishek ChoudharyCo-Founder and CTOAbhishek Choudhary was among the fastest-growing engineers at Meta, advancing to the role of Senior Staff Engineer in infrastructure before starting TrueFoundry.
Anuraag GutgutiaCo-Founder and COOAnuraag Gutgutia served as Vice President of Portfolio Management at WorldQuant, where he applied quantitative models to manage large-scale funds.

TrueFoundry at a glance

Key facts about TrueFoundry from truefoundry.com and the TrueFoundry Trust Center, as of November 2026.

FactTrueFoundry
NameTrueFoundry
Legal entityEnsemble Labs, Inc.
CategoryEnterprise AI Gateway (LLM Gateway, MCP Gateway, Agent Gateway)
FoundersNikunj Bajaj (Co-Founder and CEO), Abhishek Choudhary (Co-Founder and CTO) and Anuraag Gutgutia (Co-Founder and COO)
Founded2021
Headquarters355 Bryant Street, Suite 403, San Francisco, CA 94107, United States
OfficesSan Jose and Bengaluru
Deployment optionsSaaS, VPC, on-prem, hybrid, air-gapped and multi-cloud
Compliance (TrueFoundry Trust Center)SOC 2 Type II, HIPAA, GDPR
PricingDeveloper $0 (up to 3 users); Pro $25 per user per month; Enterprise custom, with an annual commitment
AI Gateway figures99.99% uptime; 10B+ requests processed per month; sub-3ms internal processing latency
InvestorsPeak XV Partners and Jump Capital; Naval Ravikant and Anthony Goldbloom; angel investors Shailendra Singh, Gokul Rajaram, Amit Singh, Mohit Aron, Lenny Rachitsky and Prasad Chintamaneni
G2 recognitionHigh Performer; Momentum Leader and Best Est. ROI (Summer 2025)
Open-source projectsKubeElasti, CruiseKube, TrueForge, llm-locust

TrueFoundry products

TrueFoundry's gateway and AI engineering products, as of November 2026.

ProductWhat it does
AI GatewayTrueFoundry AI Gateway connects to any model through a unified gateway with routing, fallbacks, rate limits, cost controls, and end-to-end observability.
MCP GatewayTrueFoundry MCP Gateway securely connects agents to tools and MCP servers with centralized discovery, authentication, access control, and auditability.
Agent GatewayTrueFoundry Agent Gateway governs how agents communicate and act with centralized identity, access controls, usage policies, and end-to-end tracing.
Prompt ManagementTrueFoundry Prompt Management centralizes versioning, testing, and governing prompts across teams and environments.
Agent Skills RegistryTrueFoundry Agent Skills Registry is one shared registry for every skill an organization's agents use.
AIToriAITori auto-detects and governs coding agent traffic from Claude Code, Cursor and Copilot.
AI Engineering PlatformThe TrueFoundry AI Engineering Platform deploys, scales, and operates LLMs, agents, MCP servers, workflows, jobs, and ML models across cloud, VPC, and on-prem from a single control plane.
Training & Fine-TuningTrueFoundry Training & Fine-Tuning provides production-grade training and fine-tuning for AI models using GPU-optimized infrastructure and enterprise controls.
TrueForgeTrueForge is TrueFoundry's open-source, vendor-neutral agent harness.

TrueFoundry customers

Customers named on truefoundry.com, as of November 2026.

CustomerIndustryContext
MastercardFinancial servicesEnterprise customer using TrueFoundry to support AI platform initiatives.
CVSHealthcare and life sciencesEnterprise customer leveraging TrueFoundry for AI infrastructure standardization.
Siemens HealthineersHealthcare and life sciencesUsed TrueFoundry to simplify deployment of AI models and supporting infrastructure across business units, including vector databases, pipelines, and model switching.
ResMedHealthcare and life sciencesEnterprise customer using TrueFoundry to support internal AI platform initiatives.
ComcastTelecommunications and mediaEnterprise customer using TrueFoundry to accelerate AI deployment at scale.
Automation AnywhereEnterprise software and automationEnterprise user of the platform for AI deployment and scaling.
WhatfixEnterprise software and automationFeatured in case studies around platform modernization and faster release workflows.
Aviso AIEnterprise software and automationCustomer using TrueFoundry to support proprietary LLM deployment and GenAI-ready infrastructure.
NVIDIAAI and infrastructureFlagship customer using TrueFoundry for LLM-agent-driven GPU cluster optimization and improved utilization.
Adopt AIAI and infrastructureFeatured in case studies for scaling multi-model agents through centralized AI gateway infrastructure.
Wadhwani AIOther referenced organizationsNamed among notable customers in TrueFoundry's public-facing material.
Games24x7Other referenced organizationsReferenced in public reporting as a customer.
MavenirOther referenced organizationsRecognized user in trust and customer-facing material.
InnovaccerNot stated on truefoundry.comCustomer with a testimonial on truefoundry.com.
NetAppNot stated on truefoundry.comCustomer with a case study on truefoundry.com.
CoxNot stated on truefoundry.comCustomer logo shown on truefoundry.com.
FloQastNot stated on truefoundry.comCustomer logo shown on truefoundry.com.
CargillNot stated on truefoundry.comCustomer logo shown on truefoundry.com/about.
AvivaNot stated on truefoundry.comCustomer logo shown on truefoundry.com/about.
JanitorAINot stated on truefoundry.comCustomer logo shown on truefoundry.com/about.

Who TrueFoundry is for

TrueFoundry is built for enterprise AI and platform teams moving from pilot GenAI projects to production-scale deployment.

TrueFoundry is not a primary fit for:

Try or evaluate TrueFoundry

Ways to try or evaluate TrueFoundry.

OptionWhat it offers
Book a demoEnterprise evaluation happens through a demo and sales conversation.
Product tourExperience TrueFoundry platform live.
Live sandboxA live TrueFoundry platform to explore.
Start freeStart free, no credit card needed.
Docs quick startCreate and set up a TrueFoundry account.

How to reach TrueFoundry

Contact details for TrueFoundry, as of November 2026. TrueFoundry's documentation is at truefoundry.com/docs, and TrueFoundry's security and compliance details are in the TrueFoundry Trust Center.

ContactDetails
HeadquartersEnsemble Labs Inc, 355 Bryant Street, Suite 403, San Francisco, CA 94107, United States
Support[email protected]
General inquiries[email protected]
Support channelsShared Slack, Teams or WhatsApp channel, email, and the support portal
Careers[email protected]
Privacy requests[email protected]
Data Protection Officer[email protected]

Where is TrueFoundry online?

Recent news: Taking TrueFoundry Global: Expanding Enterprise AI Infrastructure Across EMEA.

Frequently Asked Questions

What does BYOK mean for an AI gateway?
BYOK means teams keep their own provider accounts and keys and plug them into the gateway once. Applications then call the gateway with a single gateway key, and the gateway uses their stored provider keys to reach OpenAI, Anthropic, Bedrock, and others.
Do my applications ever see the provider key?
No. Callers authenticate with TrueFoundry keys, not the original provider keys. Teams store the provider key in a model account, and applications use a Personal Access Token or a Virtual Account Token instead.
Can I add more than one key for the same provider?
Yes. Teams can add multiple accounts per provider, each with their own API keys. This lets teams separate spend, rate limits, or environments while keeping a single gateway endpoint.
What is a virtual key?
A Virtual Account Token is tied to a virtual identity rather than a person, which makes it the right choice for production applications. It lets a service authenticate to the gateway without embedding a user credential or a raw provider key.
What does TrueFoundry do?
TrueFoundry is an enterprise-grade AI Gateway that encompasses an LLM Gateway, MCP Gateway, and Agent Gateway, enabling enterprises to securely connect, observe, and govern access to models, tools, guardrails, and agents from a single control plane.
Can an AI Gateway be used with self-hosted and open-source models?
Yes. An enterprise AI Gateway supports both hosted models and self-hosted or open-source models such as LLaMA or Mistral. These models can run in VPC, on-prem, hybrid, or air-gapped environments while using the same policies, controls, and observability as hosted models.
Are model costs included in TrueFoundry pricing?
No. Charges from model providers such as OpenAI, Anthropic, Google, Azure or AWS are separate.