TrueFoundry AI Gateway on VMware Cloud Foundation

Enterprise AI Governance — On Your Infrastructure, Under YourControl

The Challenge

Artificial intelligence is moving fast.Business units are experimenting with large language models (LLMs) frommultiple vendors, spinning up subscriptions independently, and routingsensitive corporate data through public cloud APIs — often with no visibility,no spending controls, and no audit trail. At the same time, regulatedindustries cannot send proprietary data outside the corporate perimeter. IT andsecurity teams are left managing an expanding, ungoverned AI surface with nopractical way to enforce policy, track costs, or ensure compliance. The result:AI sprawl, budget risk, and unacceptable data exposure.

The Solution

TrueFoundry AI Gateway provides a single, centrally governed access point for all AI modelinteractions across the enterprise. Whether a team uses an on-premises modelfor sensitive workloads or a cloud provider for general productivity, everyrequest flows through one unified endpoint — monitored, attributed, andpolicy-controlled.

VMware Cloud Foundation (VCF) delivers the enterprise-grade private cloud platform that makessovereign AI possible. Built on vSphere, vSAN, and NSX, VCF runs productionworkloads fully on-premises with the operational consistency enterprisesexpect. VCF's Kubernetes service (VKS) provides the compute foundation on whichTrueFoundry AI Gateway is deployed and managed.

Together, they deliver a production-ready,auditable AI platform that keeps every model interaction inside the customer'sown infrastructure — with no dependency on external cloud services forgovernance, logging, or policy enforcement.

 Key Capabilities

All capabilities listed below have beenvalidated on VMware Cloud Foundation with vSphere Kubernetes Service.

- Unified Model Access — A single API endpoint routes requests to any combination ofon-premises and cloud AI models. Teams use one standard interface regardless ofwhich model is deployed behind it, today or in the future.

- Complete Usage Visibility — Every request and response is logged in full, stored on-premises,and queryable in real time. AI Monitoring dashboards surface usage trends,model performance, and cost breakdowns by team, project, or individual.

- Budget and Rate Controls — Per-team and per-model spending caps are enforced at the gatewaybefore a request ever reaches a model. When a budget threshold is hit, accessis blocked automatically — no manual intervention required.

- Content Guardrails — Policy-based filters inspect every prompt before it reaches a model.Rules can block specific topics, keywords, or categories of content, givingcompliance and legal teams direct control over what the AI layer will and willnot process.

- Intelligent Response Caching — Repeated queries are answered from cache at zero token cost,reducing both API spend and response latency by up to 67 times compared to acold model call.

Business Benefits

Data SovereigntyAI traffic never leaves the corporate perimeter. All model interactions, logs,and governance policies run inside VCF. For financial services, healthcare,defense, and other regulated sectors, this is the only viable path toenterprise-grade AI adoption.

Cost Accountability Token usage is tracked at the individual request level and attributedto the team or project that generated it. Platform teams can produce accuratechargeback reports across every business unit — using their own infrastructure,with no dependency on cloud vendor billing APIs.

Operational Simplicity TrueFoundry AI Gateway deploys on existing VCF infrastructure withoutnew cloud accounts, additional SaaS vendors, or separate management planes. Oneunified platform governs all AI access from a web-based dashboard, reducingoperational overhead for IT teams already managing VCF.

Use Cases

Why VCF and TrueFoundry

VMware Cloud Foundation is the provenfoundation for enterprise private cloud — delivering the infrastructureconsistency, security, and operational scale that large organizations dependon. TrueFoundry brings the AI governance layer that enterprise IT has beenmissing: a production-grade platform for controlling, observing, andattributing every AI interaction across the organization.

The combination closes the gap between AIambition and enterprise reality. Organizations can move fast with AI — routingto cloud models for agility and on-premises models for sensitivity — whilemaintaining the data control, cost discipline, and audit capabilities thatregulated environments demand.

Validated on VMware CloudFoundation

All capabilities described in this brief havebeen deployed and validated on VMware Cloud Foundation with vSphere KubernetesService. A detailed technical reference architecture and deployment guide arein development as part of this joint initiative.

Get Started

To learn more about deploying TrueFoundry AIGateway on VMware Cloud Foundation, contact the Broadcom AI PlatformSolutions team or reach out directly to TrueFoundry at www.truefoundry.com.

© 2026 Broadcom Inc. All rights reserved.VMware Cloud Foundation and vSphere are trademarks of Broadcom Inc. TrueFoundryis a trademark of TrueFoundry, Inc.