Self-Hosted AI: On-Premises vs. VPC Deployment

“Can we self-host it?” is the first question most regulated teams ask about an AI platform, and the honest answer is “yes, but decide which kind.” Self-hosting is not a single switch. It is a spectrum, and the two choices that matter most are whether the platform runs in your cloud account (VPC) or in your own datacenter (on-premises) — with a fully air-gapped install at the strict end of the on-prem option.

They sound similar because both keep your workloads in your environment. They differ in what you operate, where your GPUs come from, how much network isolation you get, and how much of the platform you run yourself. Picking the wrong one means either taking on datacenter operations you did not need or discovering, late, that a public-cloud dependency is a compliance problem.

Self-hosted is a spectrum, not a switch

Think of deployment as a line, where each step moves more of the platform into your control and asks more of your team.

At one end, everything is managed for you (SaaS). Next, the data plane moves into your cloud account (VPC/BYOC). Then the whole thing moves into your datacenter (on-prem). At the far end, it runs with no outbound internet at all (air-gapped). “On-premises vs. VPC” is the middle of this line — the part most regulated enterprises actually choose between.

Not sure which deployment fits your compliance rules?

Book a demo

The split that makes both possible

The reason one platform can run in your cloud or your datacenter is that it separates two planes. The control plane is where you manage and monitor deployments — it can be TrueFoundry-hosted SaaS or self-hosted in your VPC. The compute plane is where your workloads actually run: one or more Kubernetes clusters in your environment.

The key architectural detail: the control plane never reaches into your clusters. A tfy-agent runs inside each cluster, connects outbound to the control plane over a secure WebSocket, receives deployment instructions, and streams workload state back. Your code, models, and data never leave your infrastructure. That one design is what lets the compute plane sit in a cloud VPC or an on-prem rack without changing how the platform works.

VPC deployment (BYOC)

In a VPC deployment — bring your own cloud — the compute plane is a managed Kubernetes cluster in your cloud account: AWS EKS, Google GKE, or Azure AKS. You provision it with OpenTofu/Terraform, or attach an existing cluster you already run. GPUs come from your cloud provider, scaling is elastic, and the managed control plane (SaaS or self-hosted) orchestrates it through the agent.

This is the sweet spot for a team already operating in the cloud. Data and workloads stay inside your account and your VPC boundary, you get cloud-native autoscaling and GPU availability, and you avoid running bare-metal infrastructure. You still own cluster hygiene — upgrades, quotas, node pools — but not a datacenter.

On-premises deployment

In an on-premises deployment, the compute plane is your own cluster — any standard Kubernetes or OpenShift cluster, including on-prem GPU clusters and neoclouds. This is the option when a hard rule says data and inference cannot touch a public cloud at all: regulated data, sovereignty requirements, or an existing GPU investment you want to use.

On-prem is also the path to air-gapped, where the environment has no outbound internet and every dependency is mirrored internally. You get the strictest possible isolation, at the cost of running the full stack yourself — the cluster, the GPUs, the storage, and the upgrade process. GPU capacity is whatever you have racked, not whatever the cloud can burst to, so capacity planning becomes your job rather than your provider’s.

Side by side

The decision usually comes down to two questions: can your data sit in a public cloud account at all, and do you want to operate hardware. If the data can live in your cloud and you would rather not run a datacenter, VPC wins. If a rule forbids the public cloud, or you need an air gap, or you already own the GPUs, on-prem is the answer.

What actually leaves your environment

The anxiety behind this decision is always about data, so concretely: in both VPC and on-prem, your code, models, and request data stay in your infrastructure, encrypted at rest and in transit. The control plane never pulls from your clusters — it only receives the workload state the tfy-agent streams out. If you also self-host the control plane, the only outbound data is what a licensing server needs (platform-user emails and request counts, for billing). In an air-gapped install, even that is handled through an internal process. Across every option, the data path is the same; what changes is how much of the platform you operate.

Which should you choose?

Start from the constraint, not the preference. If your data can live in your cloud account, VPC is the lightest self-hosted option and the one most teams should pick — you get isolation and data ownership without standing up a datacenter. Move to on-premises when a concrete rule requires it: data that cannot touch a public cloud, an air-gap mandate, data-sovereignty law, or GPUs you already own and must use. Reach for air-gapped only when “no outbound internet” is an actual requirement, because it is the most to operate. The mistake is choosing on-prem by default “to be safe” and taking on months of infrastructure work a VPC deployment would have avoided.

Run the same platform from SaaS to air-gapped, and move as your rules change.

Book a demo

Where teams get this wrong

They conflate “self-hosted” with “on-prem.” VPC is self-hosted too — your cloud, your data — with far less operational weight. Many teams that say “we need on-prem” actually need a VPC deployment.

They under-budget on-prem GPU operations. In the cloud you burst; on-prem you capacity-plan, rack, and maintain. That is real, ongoing work.

They forget the control plane is a separate choice. You can run the compute plane on-prem while the control plane stays SaaS, or self-host both. These are independent decisions.

They pick air-gapped without needing it. Air-gap means mirroring every dependency and losing managed convenience. Unless a rule requires no outbound internet, a non-air-gapped on-prem or VPC install is far cheaper to run.

Related reading

• AI gateway on-premise
• LLM gateway on-premise infrastructure
• AI gateway data residency comparison
• Air-gapped AI for regulated industries
• On-premise AI platform

Conclusion

“Self-hosted” is a decision with two real options, not one. VPC keeps your workloads in your cloud account with managed Kubernetes and cloud GPUs — the lighter path, and the right one for most teams that can keep data in a cloud they control. On-premises puts everything in your datacenter, up to a fully air-gapped install — the right path when a rule forbids the public cloud or you own the hardware.

The control-plane / compute-plane split means you do not have to trade governance for isolation: your data stays in your environment either way, and you can move along the spectrum as your requirements change without switching products.

Find the self-hosted deployment that fits your rules

Book a demo