SaaS vs Hybrid vs Self-Hosted: Choosing an AI Gateway Control Plane

The split that matters: control plane vs data plane

Every serious AI gateway separates two concerns. The data plane is the part in the live path of a request: it receives your app’s call, applies routing and guardrails, forwards it to the model provider, and returns the response. The control plane is everything around that: the dashboard, configuration, authentication, metrics, and logs.

This difference is what allows for flexible deployment. It is possible for you to maintain the performance-sensitive and latency-sensitive data plane near your workloads, and let someone else manage the management plane—or manage everything yourself. “Choosing among SaaS, hybrid, and self-hosted” boils down to one simple issue: “where do which planes run and where are requests stored?”

The spectrum

There isn’t a binary choice between “cloud” and “on-prem.” It’s a spectrum, and each step moves more into your environment.

At one end, everything is managed for you. At the other, everything runs in your VPC. In between are the options most regulated enterprises actually pick — managed convenience where it’s safe, self-hosting where a requirement demands it.

The three models, side by side

SaaS: fully managed

Both planes run on TrueFoundry’s cloud. There’s no infrastructure to deploy, the gateway is globally distributed across regions and clouds for low latency and high availability, and the platform is SOC2, ISO 27001, GDPR, and HIPAA compliant. Hosting cost to you is zero.

SaaS — both planes in TrueFoundry’s cloud. Source: TrueFoundry docs.

A middle variant keeps the managed gateway but stores request-response data in your own blob storage (S3, GCS, or Azure Blob) in Parquet — so you retain full data ownership and can run analytics on it with Spark, DuckDB, or Athena, while TrueFoundry still operates the gateway.

Hybrid: your gateway, their control plane

Here the gateway data plane runs in your VPC and TrueFoundry hosts the control plane. This is the sweet spot for many enterprises: all LLM traffic stays within your infrastructure — TrueFoundry is never in the live request path — and request data is stored in your own bucket, while you avoid running the management layer yourself.

Hybrid — the gateway runs in your environment; TrueFoundry runs the control plane. Source: TrueFoundry docs.

The trade-off is that you manage the availability of the gateway and its ingestion service, and the control plane needs read access to your log bucket so you can browse request logs in the dashboard.

Self-hosted: everything in your environment

Both the gateway and the control plane run in your infrastructure. Everything except the authentication/licensing and analytics servers is yours.

Self-hosted — both planes in your VPC. Source: TrueFoundry docs.

The only data that leaves is what the licensing server needs: the emails of platform users and the count of requests through the gateway, for license and billing tracking. This is the option for the strictest data-residency and isolation requirements — at the cost of running the full platform yourself (~$800–1,000/month in hosting).

Which should you choose?

Which deployment should you choose?

The honest default is start on SaaS. It’s the fastest path to production and, for most teams, its compliance posture is more than enough. Move the data plane into your VPC (hybrid) when you have a concrete reason — LLM traffic that must not leave your network, or request data that must live in your own storage. Go fully self-hosted when data-residency, air-gap, or regulatory rules require that even the control plane run in your environment. The mistake is choosing self-hosted by default “to be safe” and taking on months of platform operations you didn’t need.

Where teams get this wrong

They self-host out of habit, not requirement. Running both planes is real operational work. Unless a specific rule demands it, you’re paying in engineering time for control you may not need.

They conflate “managed” with “no data ownership.” Even on SaaS you can store request data in your own bucket. Managed gateway and data ownership are not mutually exclusive.

They forget egress costs. When logs live in your bucket and you browse them in the dashboard, data is fetched across the network — your cloud may charge egress. Budget for it in hybrid and self-hosted.

They underestimate ops for hybrid. Running the gateway in your VPC means owning its uptime and the ingestion service. It’s lighter than full self-hosting, but it’s not zero.

‍