Self-Hosted AI: On-Premises vs. VPC Deployment
By Ashish Dubey · Published October 17, 2026 · Updated October 17, 2026
TL;DR
- TrueFoundry runs its compute plane in a team's own cloud account (EKS, GKE or AKS) or on any standard Kubernetes or OpenShift cluster on-prem, up to fully air-gapped; the control plane can be TrueFoundry SaaS or self-hosted in the team's VPC.
- “Self-hosted” is not one thing. It splits into VPC (the platform runs in a team's own cloud account) and on-premises (it runs in the team's datacenter, up to a fully air-gapped install). Both keep the team's code, models, and data in the team's environment.
- The control-plane / compute-plane split is what makes both possible. The control plane can be TrueFoundry SaaS or self-hosted in the team's VPC; the compute plane is one or more Kubernetes clusters wherever the team's workloads run.
- VPC means managed Kubernetes in the team's cloud (EKS, GKE, or AKS), provisioned with Terraform/OpenTofu or attached to an existing cluster — cloud-native, cloud GPUs, least operational lift for a cloud team.
- On-premises means any standard Kubernetes or OpenShift cluster, including on-prem GPU clusters and neoclouds — the option for data that cannot touch a public cloud, and the path to air-gapped.
- In both, the team's data never leaves the team's infrastructure. The control plane never reaches into the team's clusters directly; a tfy-agent connects outbound over a secure WebSocket and streams back workload state.
“Can we self-host it?” is the first question most regulated teams ask about an AI platform, and the honest answer is “yes, but decide which kind.” Self-hosting is not a single switch. It is a spectrum, and the two choices that matter most are whether the platform runs in their cloud account (VPC) or in their own datacenter (on-premises), with a fully air-gapped install at the strict end of the on-prem option.
They sound similar because both keep a team's workloads in the team's environment. They differ in what teams operate, where their GPUs come from, how much network isolation teams get, and how much of the platform teams run themselves. Picking the wrong one means either taking on datacenter operations a team did not need or discovering, late, that a public-cloud dependency is a compliance problem.
Self-hosted is a spectrum, not a switch
Think of deployment as a line, where each step moves more of the platform into a team's control and asks more of the team.
| Deployment | What it means |
|---|---|
| SaaS | Fully managed |
| VPC / BYOC | The team's cloud account |
| On-premises | The team's datacenter |
| Air-gapped | No outbound internet |
At one end, everything is managed for the team (SaaS). Next, the data plane moves into the team's cloud account (VPC/BYOC). Then the whole thing moves into the team's datacenter (on-prem). At the far end, it runs with no outbound internet at all (air-gapped). “On-premises vs. VPC” is the middle of this line — the part most regulated enterprises actually choose between.
The split that makes both possible
The reason one platform can run in a team's cloud or the team's datacenter is that it separates two planes. The control plane is where teams manage and monitor deployments — it can be TrueFoundry-hosted SaaS or self-hosted in their VPC. The compute plane is where their workloads actually run: one or more Kubernetes clusters in their environment.
| Part | Details |
|---|---|
| The team | Data Scientists & ML Engineers; UI, CLI, Python SDK |
| TrueFoundry Control Plane (SaaS or self-hosted in the team's VPC) | Deployment Orchestration; Model Registry & Experiment Tracking; Monitoring, Autoscaling & Cost Optimization |
| Connection between the planes | Secure WebSocket via tfy-agent |
| The team's Compute Plane (the team's cloud account or on-prem) | Kubernetes Clusters (EKS, GKE, AKS, OpenShift, On-prem); Model APIs, Services, Jobs, LLMs, Workflows, Notebooks |
The key architectural detail: the control plane never reaches into the team's clusters. A tfy-agent runs inside each cluster, connects outbound to the control plane over a secure WebSocket, receives deployment instructions, and streams workload state back. The team's code, models, and data never leave the team's infrastructure. That one design is what lets the compute plane sit in a cloud VPC or an on-prem rack without changing how the platform works.
VPC deployment (BYOC)
In a VPC deployment — bring your own cloud — the compute plane is a managed Kubernetes cluster in the team's cloud account: AWS EKS, Google GKE, or Azure AKS. Teams provision it with OpenTofu/Terraform, or attach an existing cluster they already run. GPUs come from the team's cloud provider, scaling is elastic, and the managed control plane (SaaS or self-hosted) orchestrates it through the agent.
This is the sweet spot for a team already operating in the cloud. Data and workloads stay inside the team's account and the team's VPC boundary, teams get cloud-native autoscaling and GPU availability, and teams avoid running bare-metal infrastructure. Teams still own cluster hygiene (upgrades, quotas, node pools), but not a datacenter.
On-premises deployment
In an on-premises deployment, the compute plane is the team's own cluster — any standard Kubernetes or OpenShift cluster, including on-prem GPU clusters and neoclouds. This is the option when a hard rule says data and inference cannot touch a public cloud at all: regulated data, sovereignty requirements, or an existing GPU investment teams want to use.
On-prem is also the path to air-gapped, where the environment has no outbound internet and every dependency is mirrored internally. Teams get the strictest possible isolation, at the cost of running the full stack themselves — the cluster, the GPUs, the storage, and the upgrade process. GPU capacity is whatever teams have racked, not whatever the cloud can burst to, so capacity planning becomes their job rather than their provider’s.
| Location | Components |
|---|---|
| Truefoundry Cloud | Global Authentication / Licensing Server |
| Customer VPC: Control Plane | UI, Backend, Postgres, NATS, Ingestion Server |
| Customer VPC: alongside the Control Plane | Blob Storage (S3, GCS, Minio, Azure Container) and the Gateway |
| Outside the Customer VPC | Developer/Admin; Application (LLM Request and LLM Response through the Gateway); LLM Model |
Side by side
Both keep the team's code, models, and data inside the team's environment.
| Aspect | VPC (BYOC) | On-premises |
|---|---|---|
| Runs in | The team's cloud account | The team's datacenter |
| Kubernetes | Managed EKS / GKE / AKS | Any K8s or OpenShift |
| GPUs | From the team's cloud, elastic | The team's own, capacity-planned |
| Network | Cloud egress available | Can be fully air-gapped |
| Best when | Data can live in the team's cloud | Public cloud not allowed |
| Ops lift | Cluster hygiene only | Full stack plus hardware |
The decision usually comes down to two questions: can a team's data sit in a public cloud account at all, and do teams want to operate hardware. If the data can live in the team's cloud and teams would rather not run a datacenter, VPC wins. If a rule forbids the public cloud, or teams need an air gap, or teams already own the GPUs, on-prem is the answer.
What actually leaves your environment
The anxiety behind this decision is always about data, so concretely: in both VPC and on-prem, the team's code, models, and request data stay in the team's infrastructure, encrypted at rest and in transit. The control plane never pulls from the team's clusters — it only receives the workload state the tfy-agent streams out. If teams also self-host the control plane, the only outbound data is what a licensing server needs (platform-user emails and request counts, for billing). In an air-gapped install, even that is handled through an internal process. Across every option, the data path is the same; what changes is how much of the platform teams operate.
In both VPC and on-prem, the team's data stays put. Only workload state goes out.
| Location | What it holds or receives |
|---|---|
| The team's environment (VPC or on-prem) | Code, models and request data, encrypted at rest and in transit; the tfy-agent connects outbound only |
| Control plane | Workload state |
| Licensing server | Emails + request counts (self-hosted control plane only) |
Which should you choose?
| Constraint | Deployment |
|---|---|
| Data can live in the team's cloud account | VPC (BYOC) |
| Public cloud not allowed, sovereignty, or teams own the GPUs | On-premises |
| No outbound internet is a hard requirement | Air-gapped |
Start from the constraint, not the preference. If a team's data can live in the team's cloud account, VPC is the lightest self-hosted option and the one most teams should pick — teams get isolation and data ownership without standing up a datacenter. Move to on-premises when a concrete rule requires it: data that cannot touch a public cloud, an air-gap mandate, data-sovereignty law, or GPUs teams already own and must use. Reach for air-gapped only when “no outbound internet” is an actual requirement, because it is the most to operate. The mistake is choosing on-prem by default “to be safe” and taking on months of infrastructure work a VPC deployment would have avoided.
Where teams get this wrong
They conflate “self-hosted” with “on-prem.” VPC is self-hosted too (the team's cloud, the team's data), with far less operational weight. Many teams that say “we need on-prem” actually need a VPC deployment.
They under-budget on-prem GPU operations. In the cloud teams burst; on-prem teams capacity-plan, rack, and maintain. That is real, ongoing work.
They forget the control plane is a separate choice. Teams can run the compute plane on-prem while the control plane stays SaaS, or self-host both. These are independent decisions.
They pick air-gapped without needing it. Air-gap means mirroring every dependency and losing managed convenience. Unless a rule requires no outbound internet, a non-air-gapped on-prem or VPC install is far cheaper to run.
Related reading
- AI gateway on-premise
- LLM gateway on-premise infrastructure
- AI gateway data residency comparison
- Air-gapped AI for regulated industries
- On-premise AI platform
Conclusion
“Self-hosted” is a decision with two real options, not one. VPC keeps a team's workloads in the team's cloud account with managed Kubernetes and cloud GPUs — the lighter path, and the right one for most teams that can keep data in a cloud they control. On-premises puts everything in the team's datacenter, up to a fully air-gapped install — the right path when a rule forbids the public cloud or teams own the hardware.
The control-plane / compute-plane split means teams do not have to trade governance for isolation: their data stays in their environment either way, and teams can move along the spectrum as their requirements change without switching products.