Can You Run OpenRouter Self-Hosted? TrueFoundry vs OpenRouter on Deployment and Architecture

OpenRouter is a managed SaaS router with no self-hosted, VPC or on-prem edition at any tier, and every prompt you send it leaves your network. That is not a criticism of the product; it is the architecture. But it is the single fact that ends most enterprise evaluations of OpenRouter before the feature comparison starts, and it is the reason platform and security teams end up comparing it with an AI gateway they can run themselves.

This post is for the people who have to answer “where does it run, and where do the prompts go”: platform engineers sizing a production deployment, security leads with a data-boundary requirement, and architects who want to know what a control plane and a gateway plane actually do. We compare OpenRouter's hosted-only model with TrueFoundry AI Gateway's four deployment modes (managed SaaS, SaaS with your own storage, hybrid and fully self-hosted or air-gapped), walk through the architecture that makes those modes possible, and say plainly what you would have to operate in each one. Every claim is drawn from the two vendors' public documentation.

If you already know you need a gateway inside your own boundary, you can book a TrueFoundry demo and skip to the comparison table below.

At a glance

Why “OpenRouter self hosted” has no answer

OpenRouter runs both its control plane and its data plane on its own infrastructure. You operate neither, so there is nothing to deploy, no Kubernetes story, no stateful dependencies on your side and no upgrades to run. The customer-facing architecture is an endpoint and a key. Rather than deployment modes, what varies is the endpoint: the global openrouter.ai/api/v1 by default, and for Enterprise customers region-specific base URLs at eu.openrouter.ai and us.openrouter.ai, where requests are decrypted and processed only within that region and routed only to providers operating there. Private Models, also Enterprise, lets OpenRouter call out to a model endpoint you host, onboarded manually by their team (Models).

None of that is self-hosting. Prompts, responses and metadata all transit OpenRouter, and then transit whichever provider serves the request. OpenRouter documents three adjacent controls, and it is worth being precise about what each one does: in-region routing constrains the jurisdiction in which data is decrypted; zero-data-retention routing restricts requests to providers that do not store prompts or responses; and provider.data_collection: "deny" routes only to providers that do not collect user data. All three govern where the data goes and what is kept. None of them changes whether it leaves. Failover between the two regional endpoints is not documented.

The flip side deserves equal weight. Zero infrastructure at every scale is a real advantage, and the operational work does not disappear so much as move into your application: retry and timeout handling in the client, a monitored credit balance, and a decision about whether to keep a direct-to-provider path as a second route, because a hosted gateway is a single point in the request path.

TrueFoundry's four deployment modes

A self hosted AI gateway is usually a fork of a vendor's product: a different binary, a different feature set, a different support contract. TrueFoundry ships four modes as one product on one control plane, which is the structural difference on this topic (modes of deployment):

1. Fully managed SaaS. We host everything; zero infrastructure cost; a globally distributed gateway.
2. SaaS with customer-owned data storage. The same managed gateway, with traces and logs landing in your own blob bucket in the region you choose.
3. Hybrid. A self-hosted gateway plane in your VPC, so the data path stays inside, plus a TrueFoundry-hosted control plane. Roughly $600/month in baseline infrastructure.
4. Fully self-hosted. Control plane and gateway plane both in your VPC or on-prem, with air-gapped environments supported. Roughly $800–1,000/month in baseline infrastructure.

The modes solve different problems, and conflating them is the most common mistake we see in evaluations. Mode 2 solves data residency without solving data-path egress: prompts still transit our gateway. Mode 3 solves data-path egress, because prompts never leave your VPC, while keeping operations close to zero, because the gateway plane holds no state. Mode 4 solves everything except one disclosed callback: login and licensing transit our global auth server, which receives user email addresses and request counts and nothing from the request path. If you are already running models in your own environment, our guide to on-premise LLM deployment covers how the gateway sits alongside them.

Two honest footnotes. There is no open-source self-hosted edition; the $0 Developer tier is a SaaS tier, not a self-hostable binary, and self-hosting is an Enterprise-plan capability. And moving between modes (start on SaaS, graduate to hybrid) is implied by the shared control plane but is not yet documented as a supported migration procedure.

LLM gateway architecture: what a stateless gateway plane buys you

The reason the four modes can be one product is the split between the control plane and the gateway plane, and the reason the hybrid mode is cheap to run is what the gateway plane does not do.

The gateway plane is completely stateless. Configuration (models, keys, policies, RBAC) syncs down from the control plane over a NATS queue and is held in memory. Every request-path check (authentication, RBAC, rate limits, budgets) executes in memory, with no database call in the hot path. Logs and metrics flow back up asynchronously after the response, so a log write never blocks a request. The control plane holds all the state: Postgres for configuration and users, NATS for sync, ClickHouse backed by blob storage for logs and analytics (gateway plane architecture, control plane architecture). That is what gives the gateway roughly 3–4 ms of added latency and 350+ requests per second on a single vCPU while fronting 2,500+ LLMs through one OpenAI-compatible endpoint, and it is why multiple gateway planes can hang off one control plane for multi-VPC and multi-region deployments.

Two precision points keep this AI gateway architecture claim honest. “No external call in the request path” has documented exceptions: Redis on cache-enabled routes, and any TrueFoundry-managed guardrail attached to a route adds a network hop. And the split is not a hybrid-only feature; full SaaS runs the same architecture, so hybrid is the same product with one plane relocated rather than a fork.

The split also answers the question every platform team asks: what happens when the control plane is unreachable? The gateway keeps serving on last-synced configuration. What degrades is configuration freshness, aggregated budget accuracy and the ability to start new pods. The worst case is NATS and the control-plane backend both down, when real-time sync stops, the log pipeline halts, aggregated counters freeze and pod restarts fail. With NATS up and only the backend down, sync, aggregation and logs continue and only the UI and API are affected (deployment FAQ). OpenRouter's equivalent is simpler and harsher: control plane and data plane are one hosted system with no customer-operated fallback, so an outage there removes every provider at once, which is the failure a multi-provider router is adopted to avoid.

The gateway plane is written in Hono on Node.js and ships as the tfy-llm-gateway Helm chart for Kubernetes 1.27+ (EKS, GKE, AKS, OpenShift and on-prem documented). There are no CRDs and no operator: gateway objects live in the control plane and sync via NATS, so kubectl never touches gateway configuration. One brain, no per-cluster drift, with one honest cost: gateway policy is not GitOps-able through Kubernetes manifests, and GitOps/IaC is an Enterprise capability (deploy gateway plane).

Air-gapped and data residency: the precise claims

“Air-gapped” claims in this market come in three grades, and a buyer should know which one they are being sold. Marketing air-gapped is SaaS with private networking. Data-path air-gapped means no request data ever leaves, with a disclosed control-channel exception. Literally disconnected means no egress of any kind, with offline licensing.

TrueFoundry is firmly in the second grade. Fully self-hosted installs in restricted environments are documented, with image and chart mirroring into private registries and no public-internet dependency for serving traffic. The licence is a key stored as a Kubernetes Secret, validated at startup and sync time rather than per request; running pods keep serving indefinitely without connectivity to the licensing server, and only new pod starts need the control plane and NATS. There is no file-based offline licence, which is the measured distance to grade three. One self-reopening boundary must always travel with the air gapped LLM claim: our built-in PII guardrail is powered by Azure AI Language and runs on TrueFoundry-managed infrastructure, so enabling it in a self-hosted deployment sends request content across the boundary. Air-gapped configurations should use self-hosted or custom guardrails instead (PII guardrail).

OpenRouter does not sit on the scale. On OpenRouter data residency specifically, the sharpest finding in its documentation is that Input & Output Logging does not apply to requests routed through the EU or US endpoints; those requests are processed normally but logging is skipped. In-region processing and stored prompt content are therefore mutually exclusive, so an enterprise buyer who wants EU residency cannot also have prompt-level debugging or audit through the platform. Where OpenRouter's retained request metadata lives, and in which region, is not documented.

On multi-region: TrueFoundry SaaS fails over automatically to the nearest healthy gateway (globally distributed SaaS); self-hosted multi-region is several gateway planes under one control plane (Enterprise), with traffic steering and control-plane disaster recovery yours to design.

TrueFoundry vs OpenRouter: deployment and architecture head-to-head

TrueFoundry capabilities are from our documentation; OpenRouter capabilities reflect its publicly documented functionality at the time of writing.

What you operate, and what it costs

TrueFoundry's production floor is mode-shaped rather than scale-shaped. In hybrid mode it is a Kubernetes cluster across two availability zones, two or more stateless gateway replicas with autoscaling and anti-affinity, ingress with TLS, egress to the control plane and providers, and Redis only if you use caching or need exact cross-replica budgets. High availability really is just replicas. Moving to fully self-hosted multiplies your stateful components from one optional to four required: managed multi-AZ Postgres, a three-node NATS cluster, replicated ClickHouse and durable blob storage, each either a managed service or your pager. That is why the baselines are about $600/month for hybrid and $800–1,000/month for fully self-hosted before scaling, and why we estimate roughly 0.1–0.15 FTE for hybrid and 0.3–0.5 FTE for a self-hosted control plane (our estimates, not published numbers).

OpenRouter's line is $0 and roughly 0.02 FTE for a small team, and it is a genuine advantage. The comparison a buyer should actually make is not “free versus $800”: it is whether the requirement that sent them looking for an enterprise LLM gateway can be met by a hosted product at all. If prompts must stay inside the boundary, the infrastructure line is the price of meeting the requirement. The wider field of self-hosted options is covered in our guide to OpenRouter alternatives.

When to choose which

• Choose OpenRouter when you want no infrastructure at all and your prompts may transit a third party. Plan for it as a SaaS API with client-side resilience: retries, a monitored credit balance, and a direct-to-provider fallback for anything critical.
• Choose TrueFoundry SaaS when you want zero infrastructure today but expect to need a VPC or hybrid deployment within the next year. It is the same product on the same control plane, so the path exists even though the migration procedure is not yet formally documented.
• Choose TrueFoundry hybrid when prompts must stay in your VPC but you do not want to operate a stateful control plane. The footprint is two or more stateless replicas and optional Redis, which is the mode where the architecture pays for itself.
• Choose TrueFoundry fully self-hosted when the control plane itself must sit inside your boundary: air-gapped, regulated, or sovereignty-driven. Budget for the four-component stateful stack and a dedicated node group, use self-hosted guardrails rather than the built-in PII guardrail, and write your own control-plane disaster-recovery plan.

Related reading

• On-Premise LLM Deployment — running models and the gateway inside your own environment
• On-Premise AI Platform — what a full on-prem AI stack looks like
• OpenRouter Alternatives — the wider field of self-hosted and enterprise options
• What is an LLM Gateway? — the architectural primer
• OpenRouter vs AI Gateway — the category difference behind this comparison

Conclusion

OpenRouter self-hosted is not a product that exists, and no amount of regional routing or zero-retention configuration turns a hosted marketplace into one. OpenRouter wins the deployment heading on simplicity and loses it on choice: nothing to run, nothing you can keep inside your network. TrueFoundry wins on choice and pays for it in footprint: four modes on one control plane, a stateless gateway plane that keeps serving through control-plane outages, and a self-hosted option whose cost is a stateful stack and an engineer's time. For any organisation with a data-boundary requirement, the decision is made on this heading alone.

If you are evaluating an AI gateway for a VPC, hybrid or air-gapped deployment, book a demo to see the gateway plane running inside your own cluster, or explore the TrueFoundry AI Gateway to start on managed SaaS and move the plane later.