Migrating from Bifrost: Moving a Self-Hosted AI Gateway to a Managed One

Bifrost is a good open source AI gateway, and many teams start there for the right reasons: it is free, fast and runs anywhere. The reasons teams later move off a self-hosted AI gateway are usually operational rather than functional. Someone has to run the cluster, keep PostgreSQL highly available, upgrade through breaking releases, and answer the security review. When that work stops being worth it, the question becomes how to move without breaking every application that calls the gateway.

This guide is the plan we would follow. It maps Bifrost’s concepts to a managed gateway (we use the TrueFoundry AI Gateway as the destination), lists the client changes, and lays out a cutover with a rollback path. It also covers what does not carry over, so you can scope the work honestly.

Where we lean on our own experience, from running Bifrost OSS v2.0.0 and from installing a TrueFoundry gateway plane in our own cluster in October 2026, we say so.

When a self-hosted AI gateway stops being worth it

The free tier of Bifrost is a complete gateway, so feature gaps are rarely the trigger. These usually are:

• You need more than one node. OSS replicas don’t share state. They read the same static config.json, need a restart for every change, and keep budgets, rate-limit counters and key revocations per pod. Live sync is Enterprise only.
• Security review. SSO, RBAC, audit logs, vault-backed secrets and guardrails are Enterprise features, priced by quote. Service identities don’t exist in either tier; services hold long-lived virtual keys.
• Upgrades. The v1.5.0 release carried 16 breaking changes and database migrations that cannot be rolled back. Someone has to read every release note and back up PostgreSQL before each upgrade.
• On-call. The gateway sits in the path of every model call. If it goes down at 2 a.m., it is your page.

If two or more of these are true, a managed gateway usually costs less than the engineering time. Our posts on Bifrost pricing and [USER TO FILL: link to Bifrost Enterprise vs OSS post when live] cover the buy-versus-run math in more detail.

Pick the deployment model first

“Managed” covers more than one shape, and the choice decides how much of the migration is infrastructure work. TrueFoundry offers three (deployment options):

We used the middle option for our own evaluation. The gateway plane installs from a Helm chart and authenticates to the control plane with a gateway token; models, routing and access rules are configured in the control plane and synced to the gateway. The chart also deploys a Redis pod next to the gateway. Prerequisites are Kubernetes 1.27 or later, block-storage volumes, a domain and certificate for the gateway endpoint, an image pull secret from TrueFoundry, and egress from the cluster to your control plane URL (deploy gateway plane). Self-hosting the gateway plane is on TrueFoundry’s Enterprise tier.

Step 1: inventory what Bifrost is doing today

Most migration surprises come from config nobody remembers writing. Before creating anything in the new gateway, list:

• Providers and keys: every provider, every key, key weights, and custom endpoints such as vLLM or Ollama.
• Virtual keys: owner (team, customer or none), allowed providers and models, budgets, rate limits, MCP tool access, and expiry.
• Routing: weighted provider configs, fallback chains, retry settings and every CEL routing rule, with its scope and priority.
• MCP: connected MCP servers, tool groups and their auth modes.
• Observability: content-logging settings, OTel and other connectors, retention, and any dashboards or alerts built on bifrost_* metrics.
• Plugins: custom Go plugins, semantic caching, and the Maxim plugin.
• Callers: every application, which SDK it uses, which Bifrost route it calls (OpenAI-compatible or a provider-native route), and which x-bf-* headers it sends.

If you run Bifrost with a file-based config, config.json is most of this list. If you use the database-backed config store, read it back through the management API rather than the UI, so nothing is missed.

Step 2: map Bifrost concepts to the new gateway

The mapping below is the core of the migration. Bifrost behavior comes from Bifrost’s docs; TrueFoundry behavior comes from the TrueFoundry docs linked in each row.

Rewriting CEL routing rules

CEL rules are the part of a Bifrost config that does not translate mechanically. Sort each rule by what it is really doing:

• Splitting traffic or failing over (weighted targets, fallbacks): becomes a virtual model.
• Routing by team or environment (for example, a header like x-environment): becomes a separate virtual model per environment, with access granted to the right team, or metadata-scoped policies.
• Reacting to budget usage (moving traffic to a cheaper model past a threshold): becomes a budget rule plus a fallback, or a decision in the application.
• Anything else: write it down and decide whether it belongs in the gateway at all.

Step 3: change the clients

For an application using the OpenAI SDK, the change is three values: base URL, API key and model name.

from openai import OpenAI

# Before: Bifrost
client = OpenAI(base_url="http://bifrost.internal:8080/v1",
                api_key="unused",
                default_headers={"x-bf-vk": "sk-bf-team-search",
                                 "x-bf-dim-environment": "production"})
client.chat.completions.create(model="openai/gpt-4o-mini", messages=msgs)

# After: TrueFoundry AI Gateway
client = OpenAI(base_url=GATEWAY_BASE_URL,               # e.g. https://gateway.truefoundry.ai
                api_key=VIRTUAL_ACCOUNT_TOKEN,           # one per service
                default_headers={"X-TFY-METADATA": '{"environment": "production"}'})
client.chat.completions.create(model="openai-main/gpt-4o-mini", messages=msgs)

A few things to check per caller:

• Auth becomes mandatory. In our testing, a default Bifrost OSS install accepted requests with no credentials, so some internal callers may never have sent one. Every TrueFoundry request carries a bearer token. Find those callers before cutover, not during it.
• Provider-native routes. If a caller uses Bifrost’s Anthropic, Gemini or Bedrock-compatible routes with the provider’s own SDK, check the destination’s native SDK support. TrueFoundry supports the OpenAI, Anthropic, Google Gen AI, Azure OpenAI and boto3 SDKs, with routing configuration supported for some and not others (native SDK support).
• Response shape. When we pointed the OpenAI Python SDK at Bifrost v2.0.0, four of six scenarios matched OpenAI exactly; the streaming role, logprobs and error shape differed. Code that parses Bifrost’s error bodies or works around its streaming quirks may need adjusting when it talks to a different gateway. Test error handling, not only the happy path.
• Provider differences don’t go away. In our Bifrost tests the same request shape produced different tool-schema strictness, reasoning parameters and model-retirement errors across providers. That is true behind any gateway, so keep per-provider contract tests in your CI.

Step 4: cut over gradually, with a way back

1. Stand up the new gateway next to Bifrost. Recreate providers, models, virtual accounts, budgets in audit mode, rate limits, guardrails and logging configs. Leave Bifrost untouched.
2. Run contract tests against both gateways: non-streaming and streaming chat, tool calls, structured output, embeddings, error cases and each provider-native route you use.
3. Move one low-risk service first. Switch its base URL, token and model names through config or a feature flag, so switching back is a deploy, not a code change.
4. Compare for a week. Token counts and calculated cost per service should line up between Bifrost’s logs and the new gateway’s dashboard. A difference usually means a pricing or model-name mapping is wrong.
5. Move the rest in batches, highest-volume services last. Flip budgets from audit to enforce once numbers agree.
6. Keep Bifrost running read-only for a rollback window, then revoke its virtual keys. On multi-replica OSS, revoking a key does not propagate between pods, so restart every replica after revoking, or simply shut Bifrost down.

What does not carry over

• Request history. Bifrost’s SQLite or PostgreSQL schema is not a stable interchange format. If you need the history for audits or evals, export it through the logs API or your existing connectors before decommissioning, and keep it outside the gateway.
• Custom Go plugins. They are compiled against a specific Bifrost core and Go version, so they move as a rewrite.
• CEL expressions, as above.
• Cache contents. Semantic and exact-match caches start cold on the new gateway.
• Dashboards and alerts built on bifrost_* metric names and labels.
• Bifrost-specific headers such as x-bf-vk, x-bf-dim-*, x-bf-lh-* and the per-request logging override.
• MCP OAuth consents for per-user connections; users authorize again.

Gotchas from our own setup

These come from installing a TrueFoundry gateway plane in our cluster and pointing test traffic at it.

• Tenant-wide guardrail rules apply to new callers immediately. Our tenant already had a PII guardrail rule; our first test requests were rejected with HTTP 400 until we excluded the test virtual account from it. Review existing guardrail rules, and run new ones in audit mode, before moving production traffic.
• Request logging is on by default. With a hosted control plane, the gateway sends request logs to it. You can keep the request and response data in a bucket in your own cloud instead, and logging configs can turn body logging off or redact it per team, model or metadata. Decide this with your security team before the first production request, not after.
• Self-hosted models register like any other provider. We registered an OpenAI-compatible endpoint running in our cluster as a self-hosted model and called it as <account>/<model>. Internal vLLM or similar endpoints you reach through Bifrost today move the same way.
• Size for the extra pod. The gateway-plane chart runs Redis alongside the gateway. Include it in your capacity and cost plan.

Is a managed gateway right for you?

Stay on self-hosted Bifrost if you run a single node, have the platform engineers to own it, and don’t need SSO, audit or service identities.

Move to Bifrost Enterprise if you want to stay on Bifrost and keep operating the gateway yourself, but need clustering and the security features. You will still own the cluster, HA PostgreSQL and upgrades.

Move to a managed gateway if the operational load is the problem. Choose SaaS if your data policy allows it, or a gateway plane in your own cluster if model traffic has to stay inside your network.

Why TrueFoundry as the destination

• You choose how managed. SaaS, a gateway plane in your cluster with a hosted control plane, or fully self-hosted, including air-gapped.
• Config sync is the architecture. Gateway pods are stateless and receive config, users, virtual accounts and authorization rules from the control plane in real time; the full config is republished every 10 minutes so a missed update cannot drift, and gateways keep serving on their last config if the control plane is unreachable (gateway plane architecture). That replaces Bifrost’s restart-every-pod model on OSS.
• Service identity built in. Virtual accounts give each application its own token with scheduled rotation, a grace period and secret-store sync.
• Policy as rules. Budgets, rate limits, guardrails and logging are separate rule sets matched on who, which model and request metadata, with audit modes for safe rollout.

If you are planning a move off a self-hosted AI gateway, book a demo and bring your Bifrost config; we can walk through the mapping with you. To see the product first, start at the TrueFoundry AI Gateway page.

Related reading

• Bifrost pricing: OSS, Enterprise costs, and what teams should know
• Bifrost vs LiteLLM
• What is an LLM gateway?
• AI gateway on premise: all you need to know
• LiteLLM alternatives

Conclusion

Moving off a self-hosted AI gateway is less about the client change, which is small, and more about faithfully recreating keys, policies and routing, then cutting over one service at a time with a way back. Do the inventory first, map every virtual key to a real identity, rewrite CEL rules on purpose rather than by translation, and keep Bifrost running until the numbers agree. If you want a second pair of eyes on the mapping, book a TrueFoundry demo.