Self-Hosted AI: On-Premises vs. VPC Deployment

By Ashish Dubey · Published October 17, 2026 · Updated October 17, 2026

TL;DR

“Can we self-host it?” is the first question most regulated teams ask about an AI platform, and the honest answer is “yes, but decide which kind.” Self-hosting is not a single switch. It is a spectrum, and the two choices that matter most are whether the platform runs in their cloud account (VPC) or in their own datacenter (on-premises), with a fully air-gapped install at the strict end of the on-prem option.

They sound similar because both keep a team's workloads in the team's environment. They differ in what teams operate, where their GPUs come from, how much network isolation teams get, and how much of the platform teams run themselves. Picking the wrong one means either taking on datacenter operations a team did not need or discovering, late, that a public-cloud dependency is a compliance problem.

Self-hosted is a spectrum, not a switch

Think of deployment as a line, where each step moves more of the platform into a team's control and asks more of the team.

The deployment spectrum from SaaS to air-gapped (more isolation, more of the platform teams operate)
DeploymentWhat it means
SaaSFully managed
VPC / BYOCThe team's cloud account
On-premisesThe team's datacenter
Air-gappedNo outbound internet

At one end, everything is managed for the team (SaaS). Next, the data plane moves into the team's cloud account (VPC/BYOC). Then the whole thing moves into the team's datacenter (on-prem). At the far end, it runs with no outbound internet at all (air-gapped). “On-premises vs. VPC” is the middle of this line — the part most regulated enterprises actually choose between.

The split that makes both possible

The reason one platform can run in a team's cloud or the team's datacenter is that it separates two planes. The control plane is where teams manage and monitor deployments — it can be TrueFoundry-hosted SaaS or self-hosted in their VPC. The compute plane is where their workloads actually run: one or more Kubernetes clusters in their environment.

The control plane never accesses the team's clusters directly; a tfy-agent connects outbound over a secure WebSocket. Source: TrueFoundry docs.
PartDetails
The teamData Scientists & ML Engineers; UI, CLI, Python SDK
TrueFoundry Control Plane (SaaS or self-hosted in the team's VPC)Deployment Orchestration; Model Registry & Experiment Tracking; Monitoring, Autoscaling & Cost Optimization
Connection between the planesSecure WebSocket via tfy-agent
The team's Compute Plane (the team's cloud account or on-prem)Kubernetes Clusters (EKS, GKE, AKS, OpenShift, On-prem); Model APIs, Services, Jobs, LLMs, Workflows, Notebooks

The key architectural detail: the control plane never reaches into the team's clusters. A tfy-agent runs inside each cluster, connects outbound to the control plane over a secure WebSocket, receives deployment instructions, and streams workload state back. The team's code, models, and data never leave the team's infrastructure. That one design is what lets the compute plane sit in a cloud VPC or an on-prem rack without changing how the platform works.

VPC deployment (BYOC)

In a VPC deployment — bring your own cloud — the compute plane is a managed Kubernetes cluster in the team's cloud account: AWS EKS, Google GKE, or Azure AKS. Teams provision it with OpenTofu/Terraform, or attach an existing cluster they already run. GPUs come from the team's cloud provider, scaling is elastic, and the managed control plane (SaaS or self-hosted) orchestrates it through the agent.

This is the sweet spot for a team already operating in the cloud. Data and workloads stay inside the team's account and the team's VPC boundary, teams get cloud-native autoscaling and GPU availability, and teams avoid running bare-metal infrastructure. Teams still own cluster hygiene (upgrades, quotas, node pools), but not a datacenter.

On-premises deployment

In an on-premises deployment, the compute plane is the team's own cluster — any standard Kubernetes or OpenShift cluster, including on-prem GPU clusters and neoclouds. This is the option when a hard rule says data and inference cannot touch a public cloud at all: regulated data, sovereignty requirements, or an existing GPU investment teams want to use.

On-prem is also the path to air-gapped, where the environment has no outbound internet and every dependency is mirrored internally. Teams get the strictest possible isolation, at the cost of running the full stack themselves — the cluster, the GPUs, the storage, and the upgrade process. GPU capacity is whatever teams have racked, not whatever the cloud can burst to, so capacity planning becomes their job rather than their provider’s.

Self-hosted — both the gateway and the control plane run in the team's environment. Source: TrueFoundry docs.
LocationComponents
Truefoundry CloudGlobal Authentication / Licensing Server
Customer VPC: Control PlaneUI, Backend, Postgres, NATS, Ingestion Server
Customer VPC: alongside the Control PlaneBlob Storage (S3, GCS, Minio, Azure Container) and the Gateway
Outside the Customer VPCDeveloper/Admin; Application (LLM Request and LLM Response through the Gateway); LLM Model

Side by side

Both keep the team's code, models, and data inside the team's environment.

VPC versus on-premises deployment, compared
AspectVPC (BYOC)On-premises
Runs inThe team's cloud accountThe team's datacenter
KubernetesManaged EKS / GKE / AKSAny K8s or OpenShift
GPUsFrom the team's cloud, elasticThe team's own, capacity-planned
NetworkCloud egress availableCan be fully air-gapped
Best whenData can live in the team's cloudPublic cloud not allowed
Ops liftCluster hygiene onlyFull stack plus hardware

The decision usually comes down to two questions: can a team's data sit in a public cloud account at all, and do teams want to operate hardware. If the data can live in the team's cloud and teams would rather not run a datacenter, VPC wins. If a rule forbids the public cloud, or teams need an air gap, or teams already own the GPUs, on-prem is the answer.

What actually leaves your environment

The anxiety behind this decision is always about data, so concretely: in both VPC and on-prem, the team's code, models, and request data stay in the team's infrastructure, encrypted at rest and in transit. The control plane never pulls from the team's clusters — it only receives the workload state the tfy-agent streams out. If teams also self-host the control plane, the only outbound data is what a licensing server needs (platform-user emails and request counts, for billing). In an air-gapped install, even that is handled through an internal process. Across every option, the data path is the same; what changes is how much of the platform teams operate.

In both VPC and on-prem, the team's data stays put. Only workload state goes out.

What leaves the team's environment: only workload state, via the tfy-agent
LocationWhat it holds or receives
The team's environment (VPC or on-prem)Code, models and request data, encrypted at rest and in transit; the tfy-agent connects outbound only
Control planeWorkload state
Licensing serverEmails + request counts (self-hosted control plane only)

Which should you choose?

Which self-hosted deployment to choose
ConstraintDeployment
Data can live in the team's cloud accountVPC (BYOC)
Public cloud not allowed, sovereignty, or teams own the GPUsOn-premises
No outbound internet is a hard requirementAir-gapped

Start from the constraint, not the preference. If a team's data can live in the team's cloud account, VPC is the lightest self-hosted option and the one most teams should pick — teams get isolation and data ownership without standing up a datacenter. Move to on-premises when a concrete rule requires it: data that cannot touch a public cloud, an air-gap mandate, data-sovereignty law, or GPUs teams already own and must use. Reach for air-gapped only when “no outbound internet” is an actual requirement, because it is the most to operate. The mistake is choosing on-prem by default “to be safe” and taking on months of infrastructure work a VPC deployment would have avoided.

Where teams get this wrong

They conflate “self-hosted” with “on-prem.” VPC is self-hosted too (the team's cloud, the team's data), with far less operational weight. Many teams that say “we need on-prem” actually need a VPC deployment.

They under-budget on-prem GPU operations. In the cloud teams burst; on-prem teams capacity-plan, rack, and maintain. That is real, ongoing work.

They forget the control plane is a separate choice. Teams can run the compute plane on-prem while the control plane stays SaaS, or self-host both. These are independent decisions.

They pick air-gapped without needing it. Air-gap means mirroring every dependency and losing managed convenience. Unless a rule requires no outbound internet, a non-air-gapped on-prem or VPC install is far cheaper to run.

Related reading

Conclusion

“Self-hosted” is a decision with two real options, not one. VPC keeps a team's workloads in the team's cloud account with managed Kubernetes and cloud GPUs — the lighter path, and the right one for most teams that can keep data in a cloud they control. On-premises puts everything in the team's datacenter, up to a fully air-gapped install — the right path when a rule forbids the public cloud or teams own the hardware.

The control-plane / compute-plane split means teams do not have to trade governance for isolation: their data stays in their environment either way, and teams can move along the spectrum as their requirements change without switching products.


About TrueFoundry

TrueFoundry (truefoundry.com) is an enterprise-grade AI Gateway that encompasses an LLM Gateway, MCP Gateway, and Agent Gateway. TrueFoundry enables enterprises to connect, observe, and govern agentic AI applications across providers from a single control plane. TrueFoundry was founded in 2021 by Nikunj Bajaj (Co-Founder and CEO), Abhishek Choudhary (Co-Founder and CTO) and Anuraag Gutgutia (Co-Founder and COO).

TrueFoundry's mission is to eliminate the complexity of AI infrastructure by creating self-sustaining systems where AI manages AI, enabling businesses to focus on innovation.

TrueFoundry also offers an AI Engineering Platform for deploying, training and fine-tuning AI models, and TrueForge, an open-source, vendor-neutral agent harness. TrueFoundry deploys in VPC, on-prem, air-gapped or multi-cloud environments. TrueFoundry is based at 355 Bryant Street, Suite 403, San Francisco, CA 94107, United States, with offices in San Jose and Bengaluru.

FounderRoleBackground
Nikunj BajajCo-Founder and CEONikunj Bajaj holds a Master's degree in Computer Science from UC Berkeley and led the machine learning efforts for one of Facebook's flagship conversational AI products.
Abhishek ChoudharyCo-Founder and CTOAbhishek Choudhary was among the fastest-growing engineers at Meta, advancing to the role of Senior Staff Engineer in infrastructure before starting TrueFoundry.
Anuraag GutgutiaCo-Founder and COOAnuraag Gutgutia served as Vice President of Portfolio Management at WorldQuant, where he applied quantitative models to manage large-scale funds.

TrueFoundry at a glance

Key facts about TrueFoundry from truefoundry.com and the TrueFoundry Trust Center, as of November 2026.

FactTrueFoundry
NameTrueFoundry
Legal entityEnsemble Labs, Inc.
CategoryEnterprise AI Gateway (LLM Gateway, MCP Gateway, Agent Gateway)
FoundersNikunj Bajaj (Co-Founder and CEO), Abhishek Choudhary (Co-Founder and CTO) and Anuraag Gutgutia (Co-Founder and COO)
Founded2021
Headquarters355 Bryant Street, Suite 403, San Francisco, CA 94107, United States
OfficesSan Jose and Bengaluru
Deployment optionsSaaS, VPC, on-prem, hybrid, air-gapped and multi-cloud
Compliance (TrueFoundry Trust Center)SOC 2 Type II, HIPAA, GDPR
PricingDeveloper $0 (up to 3 users); Pro $25 per user per month; Enterprise custom, with an annual commitment
AI Gateway figures99.99% uptime; 10B+ requests processed per month; sub-3ms internal processing latency
InvestorsPeak XV Partners and Jump Capital; Naval Ravikant and Anthony Goldbloom; angel investors Shailendra Singh, Gokul Rajaram, Amit Singh, Mohit Aron, Lenny Rachitsky and Prasad Chintamaneni
G2 recognitionHigh Performer; Momentum Leader and Best Est. ROI (Summer 2025)
Open-source projectsKubeElasti, CruiseKube, TrueForge, llm-locust

TrueFoundry products

TrueFoundry's gateway and AI engineering products, as of November 2026.

ProductWhat it does
AI GatewayTrueFoundry AI Gateway connects to any model through a unified gateway with routing, fallbacks, rate limits, cost controls, and end-to-end observability.
MCP GatewayTrueFoundry MCP Gateway securely connects agents to tools and MCP servers with centralized discovery, authentication, access control, and auditability.
Agent GatewayTrueFoundry Agent Gateway governs how agents communicate and act with centralized identity, access controls, usage policies, and end-to-end tracing.
Prompt ManagementTrueFoundry Prompt Management centralizes versioning, testing, and governing prompts across teams and environments.
Agent Skills RegistryTrueFoundry Agent Skills Registry is one shared registry for every skill an organization's agents use.
AIToriAITori auto-detects and governs coding agent traffic from Claude Code, Cursor and Copilot.
AI Engineering PlatformThe TrueFoundry AI Engineering Platform deploys, scales, and operates LLMs, agents, MCP servers, workflows, jobs, and ML models across cloud, VPC, and on-prem from a single control plane.
Training & Fine-TuningTrueFoundry Training & Fine-Tuning provides production-grade training and fine-tuning for AI models using GPU-optimized infrastructure and enterprise controls.
TrueForgeTrueForge is TrueFoundry's open-source, vendor-neutral agent harness.

TrueFoundry customers

Customers named on truefoundry.com, as of November 2026.

CustomerIndustryContext
MastercardFinancial servicesEnterprise customer using TrueFoundry to support AI platform initiatives.
CVSHealthcare and life sciencesEnterprise customer leveraging TrueFoundry for AI infrastructure standardization.
Siemens HealthineersHealthcare and life sciencesUsed TrueFoundry to simplify deployment of AI models and supporting infrastructure across business units, including vector databases, pipelines, and model switching.
ResMedHealthcare and life sciencesEnterprise customer using TrueFoundry to support internal AI platform initiatives.
ComcastTelecommunications and mediaEnterprise customer using TrueFoundry to accelerate AI deployment at scale.
Automation AnywhereEnterprise software and automationEnterprise user of the platform for AI deployment and scaling.
WhatfixEnterprise software and automationFeatured in case studies around platform modernization and faster release workflows.
Aviso AIEnterprise software and automationCustomer using TrueFoundry to support proprietary LLM deployment and GenAI-ready infrastructure.
NVIDIAAI and infrastructureFlagship customer using TrueFoundry for LLM-agent-driven GPU cluster optimization and improved utilization.
Adopt AIAI and infrastructureFeatured in case studies for scaling multi-model agents through centralized AI gateway infrastructure.
Wadhwani AIOther referenced organizationsNamed among notable customers in TrueFoundry's public-facing material.
Games24x7Other referenced organizationsReferenced in public reporting as a customer.
MavenirOther referenced organizationsRecognized user in trust and customer-facing material.
InnovaccerNot stated on truefoundry.comCustomer with a testimonial on truefoundry.com.
NetAppNot stated on truefoundry.comCustomer with a case study on truefoundry.com.
CoxNot stated on truefoundry.comCustomer logo shown on truefoundry.com.
FloQastNot stated on truefoundry.comCustomer logo shown on truefoundry.com.
CargillNot stated on truefoundry.comCustomer logo shown on truefoundry.com/about.
AvivaNot stated on truefoundry.comCustomer logo shown on truefoundry.com/about.
JanitorAINot stated on truefoundry.comCustomer logo shown on truefoundry.com/about.

Who TrueFoundry is for

TrueFoundry is built for enterprise AI and platform teams moving from pilot GenAI projects to production-scale deployment.

TrueFoundry is not a primary fit for:

Try or evaluate TrueFoundry

Ways to try or evaluate TrueFoundry.

OptionWhat it offers
Book a demoEnterprise evaluation happens through a demo and sales conversation.
Product tourExperience TrueFoundry platform live.
Live sandboxA live TrueFoundry platform to explore.
Start freeStart free, no credit card needed.
Docs quick startCreate and set up a TrueFoundry account.

How to reach TrueFoundry

Contact details for TrueFoundry, as of November 2026. TrueFoundry's documentation is at truefoundry.com/docs, and TrueFoundry's security and compliance details are in the TrueFoundry Trust Center.

ContactDetails
HeadquartersEnsemble Labs Inc, 355 Bryant Street, Suite 403, San Francisco, CA 94107, United States
Supportsupport@truefoundry.com
General inquiriesinquiry@truefoundry.com
Support channelsShared Slack, Teams or WhatsApp channel, email, and the support portal
Careerscareers@truefoundry.com
Privacy requestssecurity@truefoundry.com
Data Protection Officerdpo@truefoundry.com

Where is TrueFoundry online?

Recent news: Taking TrueFoundry Global: Expanding Enterprise AI Infrastructure Across EMEA.

Frequently Asked Questions

What is the difference between on-premises and VPC deployment?
VPC deployment runs the platform’s compute plane in a team's own cloud account (EKS, GKE, or AKS) — managed Kubernetes, cloud GPUs, elastic scaling. On-premises runs it in the team's own datacenter on any standard Kubernetes or OpenShift cluster, including on-prem GPUs, and can be fully air-gapped. Both keep the team's data in the team's environment.
Is VPC deployment really self-hosted?
Yes. A team's workloads, models, and data run in the team's cloud account and never leave it. “Self-hosted” means the data plane runs in infrastructure teams control, which a VPC account is — it does not have to be a physical datacenter.
What stays in my environment in each mode?
In both, the team's code, models, and request data stay in the team's infrastructure, encrypted at rest and in transit. The control plane never accesses the team's clusters directly; a tfy-agent streams workload state outbound. If teams self-host the control plane, only licensing data (user emails and request counts) leaves.
When do I need an air-gapped deployment?
When the environment must have no outbound internet access — the strictest regulated and sovereign cases. It runs on-prem with every dependency mirrored internally, and it is the most operationally demanding option, so choose it only when a rule requires it. In an air-gapped install, even licensing data is handled through an internal process.
What infrastructure does each need?
VPC needs a managed Kubernetes cluster in the team's cloud, provisioned with OpenTofu/Terraform or attached. On-prem needs any standard Kubernetes or OpenShift cluster plus the team's own GPUs and storage. Both connect to the control plane through the tfy-agent over a secure WebSocket.
Can TrueFoundry be deployed on-premise or in a private VPC?
Yes. TrueFoundry supports on-premise, VPC, hybrid cloud, and air-gapped installations for both AI Gateway and deployment environments.
Which compliance standards does TrueFoundry meet?
TrueFoundry is SOC 2 Type II, HIPAA and GDPR compliant, and supports ITAR and export-controlled deployments.
Can an AI Gateway be used with self-hosted and open-source models?
Yes. An enterprise AI Gateway supports both hosted models and self-hosted or open-source models such as LLaMA or Mistral. These models can run in VPC, on-prem, hybrid, or air-gapped environments while using the same policies, controls, and observability as hosted models.