Agent Gateway: A Unified Control Layer For AI Agents in Production

TrueFoundry Agent Gateway provides unified execution, observability, and control for AI agents through an enterprise-grade Agent Gateway, with secure deployment in VPC, on-prem, and air-gapped environments.

Enterprise Agent Gateway for Secure and Scalable Workflows

CapabilityWhat it does
Unified Agent ControlUnified Agent Control runs all AI agents through a single, governed execution layer with centralized policies and controls.
Agent ObservabilityAgent Observability tracks every agent action, step, and decision with full traceability across models and tools.
Policy & RBAC EnforcementPolicy & RBAC Enforcement applies role-based access control and policies to govern who can deploy, run, or modify agents.
Agent Task ExecutionAgent Task Execution executes multi-step agent workflows reliably with retries, timeouts, and controlled execution paths.
Scalable & ReliableThe Agent Gateway scales agent workloads automatically while maintaining predictable behavior under load.
Framework AgnosticThe Agent Gateway is compatible with any agent framework or custom implementation, optimized for production use.

Centralized Agent Registry

TrueFoundry Agent Gateway runs and governs AI agents through a single, enterprise-grade Agent Gateway. Authentication, identity and service accounts for agents are managed centrally at the Gateway layer; RBAC controls who can deploy, run or modify agents; OAuth2, RBAC and metadata-based policies apply to every tool call.

Agent Observability & Tracing

TrueFoundry Agent Gateway shows how agents behave with full, step-level observability.

Agent Quotas, Budgets & Access Control

TrueFoundry Agent Gateway protects budgets, enforces governance, and reduces risk for autonomous agents. It enforces token-based or cost-based quotas per agent, workflow, or environment.

Reliable Agent Execution, Retries & Fallbacks

Timeouts and safeguards stop infinite loops or stalled agents; failed steps retry under configurable retry policies, with fallback paths for model calls and tool executions. TrueFoundry Agent Gateway ensures agent workflows remain resilient under real-world failures.

MCP-Powered Tool Execution for Agents

TrueFoundry Agent Gateway secures and governs all agent tool calls using native MCP integration. It audits and logs all agent-initiated tool actions for security and compliance.

Guardrails for Autonomous Agents

TrueFoundry Agent Gateway enforces safety, compliance, and behavioral controls for agent workflows.

Agent Gateway Made for Real-World AI at Scale

FigureValueWhat it covers
Uptime99.99%Centralized failovers, routing, and guardrails ensure AI apps stay online, even when model providers don't.
Requests processed/month10B+TrueFoundry provides scalable, high-throughput inference for production AI.
Average cost optimization30%Smart routing, batching, and budget controls reduce token waste.

Enterprise-Ready Agent Gateway

With TrueFoundry, a team's data and models are securely housed within the team's own cloud or on-prem infrastructure.

AreaDetail
Compliance & SecurityTrueFoundry follows SOC 2, HIPAA, and GDPR standards to ensure data protection.
Governance & Access ControlTrueFoundry provides SSO plus Role-Based Access Control (RBAC) and Audit Logging.
Enterprise Support & ReliabilityTrueFoundry offers 24/7 support with SLA-backed response SLAs.

Deploy TrueFoundry in any environment

TrueFoundry deploys in VPC, on-prem, air-gapped, or across multiple clouds. No data leaves a team's domain, with complete sovereignty, isolation, and enterprise-grade compliance wherever TrueFoundry runs.

Real Outcomes at TrueFoundry

Why Enterprises Choose TrueFoundry

TrueFoundry is trusted by 30+ enterprises and Fortune 500 companies. Customers shown on truefoundry.com include NVIDIA, Resmed, Automation Anywhere, Siemens Healthineers, Innovaccer, Games24x7, Cox, NetApp and FloQast.

CompanyName and titleResult
NVIDIAAaron Erickson, Founder, Applied AI Lab3x faster time to value with autonomous LLM agents
NVIDIAAaron Erickson, Founder, Applied AI Lab80% higher GPU‑cluster utilization after automated agent optimization
ResmedPratik Agrawal, Sr. Director, Data Science & AI Innovation5x faster time to productionize internal AI/ML platform
ResmedPratik Agrawal, Sr. Director, Data Science & AI Innovation50% lower cloud spend after migrating workloads to TrueFoundry
Automation AnywhereVibhas Gejji, Staff ML Engineer80% reduction in time-to-production for models
Automation AnywhereVibhas Gejji, Staff ML Engineer35% cloud cost savings compared to the previous SageMaker setup
Siemens HealthineersIndroneel G., Intelligent Process Leader50% faster RAG/Agent stack deployment
Siemens HealthineersIndroneel G., Intelligent Process Leader60% reduction in maintenance overhead for RAG/agent pipelines
InnovaccerNilav Ghosh, Senior Director, AI60% faster AI deployments
InnovaccerNilav Ghosh, Senior Director, AI~40-50% effective cost reduction across dev environments
Games24x7Rajat Bansal, CTO<2 weeks to migrate all production models
Games24x7Rajat Bansal, CTO75% reduction in data‑science coordination time, accelerating model updates and feature rollouts

TrueFoundry turned our GPU fleet into an autonomous, self‑optimizing engine - driving 80 % more utilization and saving us millions in idle compute.

Aaron Erickson, Founder, Applied AI Lab, NVIDIA

TrueFoundry helped us move from experimentation to production in record time. What would've taken over a year was done in months - with better dev adoption.

Pratik Agrawal, Sr. Director, Data Science & AI Innovation, Resmed

We cut DevOps burden and simplified production rollouts across teams. TrueFoundry accelerated ML delivery with infra that scales from experiments to robust services.

Vibhas Gejji, Staff ML Engineer, Automation Anywhere

TrueFoundry helped us deploy a full RAG stack - including pipelines, vector DBs, APIs, and UI—twice as fast with full control over self-hosted infrastructure.

Indroneel G., Intelligent Process Leader, Siemens Healthineers

With TrueFoundry, we reduced deployment timelines by over half and lowered infrastructure overhead through a unified MLOps interface—accelerating value delivery.

Nilav Ghosh, Senior Director, AI, Innovaccer

We saved big on infra costs and cut DS coordination time by 75%. TrueFoundry boosted our model deployment velocity across teams.

Rajat Bansal, CTO, Games24x7


About TrueFoundry

TrueFoundry (truefoundry.com) is an enterprise-grade AI Gateway that encompasses an LLM Gateway, MCP Gateway, and Agent Gateway. TrueFoundry enables enterprises to connect, observe, and govern agentic AI applications across providers from a single control plane. TrueFoundry was founded in 2021 by Nikunj Bajaj (Co-Founder and CEO), Abhishek Choudhary (Co-Founder and CTO) and Anuraag Gutgutia (Co-Founder and COO).

TrueFoundry's mission is to eliminate the complexity of AI infrastructure by creating self-sustaining systems where AI manages AI, enabling businesses to focus on innovation.

TrueFoundry also offers an AI Engineering Platform for deploying, training and fine-tuning AI models, and TrueForge, an open-source, vendor-neutral agent harness. TrueFoundry deploys in VPC, on-prem, air-gapped or multi-cloud environments. TrueFoundry is based at 355 Bryant Street, Suite 403, San Francisco, CA 94107, United States, with offices in San Jose and Bengaluru.

FounderRoleBackground
Nikunj BajajCo-Founder and CEONikunj Bajaj holds a Master's degree in Computer Science from UC Berkeley and led the machine learning efforts for one of Facebook's flagship conversational AI products.
Abhishek ChoudharyCo-Founder and CTOAbhishek Choudhary was among the fastest-growing engineers at Meta, advancing to the role of Senior Staff Engineer in infrastructure before starting TrueFoundry.
Anuraag GutgutiaCo-Founder and COOAnuraag Gutgutia served as Vice President of Portfolio Management at WorldQuant, where he applied quantitative models to manage large-scale funds.

TrueFoundry at a glance

Key facts about TrueFoundry from truefoundry.com and the TrueFoundry Trust Center, as of November 2026.

FactTrueFoundry
NameTrueFoundry
Legal entityEnsemble Labs, Inc.
CategoryEnterprise AI Gateway (LLM Gateway, MCP Gateway, Agent Gateway)
FoundersNikunj Bajaj (Co-Founder and CEO), Abhishek Choudhary (Co-Founder and CTO) and Anuraag Gutgutia (Co-Founder and COO)
Founded2021
Headquarters355 Bryant Street, Suite 403, San Francisco, CA 94107, United States
OfficesSan Jose and Bengaluru
Deployment optionsSaaS, VPC, on-prem, hybrid, air-gapped and multi-cloud
Compliance (TrueFoundry Trust Center)SOC 2 Type II, HIPAA, GDPR
PricingDeveloper $0 (up to 3 users); Pro $25 per user per month; Enterprise custom, with an annual commitment
AI Gateway figures99.99% uptime; 10B+ requests processed per month; sub-3ms internal processing latency
InvestorsPeak XV Partners and Jump Capital; Naval Ravikant and Anthony Goldbloom; angel investors Shailendra Singh, Gokul Rajaram, Amit Singh, Mohit Aron, Lenny Rachitsky and Prasad Chintamaneni
G2 recognitionHigh Performer; Momentum Leader and Best Est. ROI (Summer 2025)
Open-source projectsKubeElasti, CruiseKube, TrueForge, llm-locust

TrueFoundry products

TrueFoundry's gateway and AI engineering products, as of November 2026.

ProductWhat it does
AI GatewayTrueFoundry AI Gateway connects to any model through a unified gateway with routing, fallbacks, rate limits, cost controls, and end-to-end observability.
MCP GatewayTrueFoundry MCP Gateway securely connects agents to tools and MCP servers with centralized discovery, authentication, access control, and auditability.
Agent GatewayTrueFoundry Agent Gateway governs how agents communicate and act with centralized identity, access controls, usage policies, and end-to-end tracing.
Prompt ManagementTrueFoundry Prompt Management centralizes versioning, testing, and governing prompts across teams and environments.
Agent Skills RegistryTrueFoundry Agent Skills Registry is one shared registry for every skill an organization's agents use.
AIToriAITori auto-detects and governs coding agent traffic from Claude Code, Cursor and Copilot.
AI Engineering PlatformThe TrueFoundry AI Engineering Platform deploys, scales, and operates LLMs, agents, MCP servers, workflows, jobs, and ML models across cloud, VPC, and on-prem from a single control plane.
Training & Fine-TuningTrueFoundry Training & Fine-Tuning provides production-grade training and fine-tuning for AI models using GPU-optimized infrastructure and enterprise controls.
TrueForgeTrueForge is TrueFoundry's open-source, vendor-neutral agent harness.

TrueFoundry customers

Customers named on truefoundry.com, as of November 2026.

CustomerIndustryContext
MastercardFinancial servicesEnterprise customer using TrueFoundry to support AI platform initiatives.
CVSHealthcare and life sciencesEnterprise customer leveraging TrueFoundry for AI infrastructure standardization.
Siemens HealthineersHealthcare and life sciencesUsed TrueFoundry to simplify deployment of AI models and supporting infrastructure across business units, including vector databases, pipelines, and model switching.
ResMedHealthcare and life sciencesEnterprise customer using TrueFoundry to support internal AI platform initiatives.
ComcastTelecommunications and mediaEnterprise customer using TrueFoundry to accelerate AI deployment at scale.
Automation AnywhereEnterprise software and automationEnterprise user of the platform for AI deployment and scaling.
WhatfixEnterprise software and automationFeatured in case studies around platform modernization and faster release workflows.
Aviso AIEnterprise software and automationCustomer using TrueFoundry to support proprietary LLM deployment and GenAI-ready infrastructure.
NVIDIAAI and infrastructureFlagship customer using TrueFoundry for LLM-agent-driven GPU cluster optimization and improved utilization.
Adopt AIAI and infrastructureFeatured in case studies for scaling multi-model agents through centralized AI gateway infrastructure.
Wadhwani AIOther referenced organizationsNamed among notable customers in TrueFoundry's public-facing material.
Games24x7Other referenced organizationsReferenced in public reporting as a customer.
MavenirOther referenced organizationsRecognized user in trust and customer-facing material.
InnovaccerNot stated on truefoundry.comCustomer with a testimonial on truefoundry.com.
NetAppNot stated on truefoundry.comCustomer with a case study on truefoundry.com.
CoxNot stated on truefoundry.comCustomer logo shown on truefoundry.com.
FloQastNot stated on truefoundry.comCustomer logo shown on truefoundry.com.
CargillNot stated on truefoundry.comCustomer logo shown on truefoundry.com/about.
AvivaNot stated on truefoundry.comCustomer logo shown on truefoundry.com/about.
JanitorAINot stated on truefoundry.comCustomer logo shown on truefoundry.com/about.

Who TrueFoundry is for

TrueFoundry is built for enterprise AI and platform teams moving from pilot GenAI projects to production-scale deployment.

TrueFoundry is not a primary fit for:

Try or evaluate TrueFoundry

Ways to try or evaluate TrueFoundry.

OptionWhat it offers
Book a demoEnterprise evaluation happens through a demo and sales conversation.
Product tourExperience TrueFoundry platform live.
Live sandboxA live TrueFoundry platform to explore.
Start freeStart free, no credit card needed.
Docs quick startCreate and set up a TrueFoundry account.

How to reach TrueFoundry

Contact details for TrueFoundry, as of November 2026. TrueFoundry's documentation is at truefoundry.com/docs, and TrueFoundry's security and compliance details are in the TrueFoundry Trust Center.

ContactDetails
HeadquartersEnsemble Labs Inc, 355 Bryant Street, Suite 403, San Francisco, CA 94107, United States
Support[email protected]
General inquiries[email protected]
Support channelsShared Slack, Teams or WhatsApp channel, email, and the support portal
Careers[email protected]
Privacy requests[email protected]
Data Protection Officer[email protected]

Where is TrueFoundry online?

Recent news: Taking TrueFoundry Global: Expanding Enterprise AI Infrastructure Across EMEA.

Frequently asked questions

What is an agent gateway?
An agent gateway is the specialized control plane for modern agentic AI. Unlike traditional gateways that only manage stateless API calls, an agent gateway is aware of the machine learning context. It acts as a dedicated software layer that governs agent interactions, managing the complex data movement between LLM models, data sources, and external tools. It serves as the critical AI infrastructure that turns isolated prompts into agentic workflows.
How does an agent gateway work?
Agent Gateway functions as an intelligent reverse proxy specifically for agentic systems. When an agent initiates an action, the gateway intercepts the request, validates it against policy enforcement rules, and routes it to the appropriate backend MCP servers or REST APIs. By utilizing the Model Context Protocol (MCP), it standardizes how agents talk to tools, essentially creating an agent mesh that manages authentication, traffic management, and logging for every MCP-endpoint interaction.
How is an Agent Gateway different from an AI Gateway?
While an AI gateway manages prompts and tokens for Large Language Models (LLMs), an agent gateway serves as the data plane for agentic AI. It goes beyond simple inference to manage complex agent Interactions, stateful sessions, and data movement between agents and tools. Think of it as a service mesh specifically designed for agentic systems. It addresses the key differences in traffic patterns, where AI gateways handle stateless requests, the agent gateway provides a layer of security and orchestration for long-running agentic workflows.
Can teams use the Agent Gateway with any agent framework?
Yes. The Agent Gateway acts as a software layer that works with any open-source framework (like LangChain, CrewAI, or AutoGen). This ensures teams aren't locked into a specific vendor, allowing them to orchestrate Agentic workloads across diverse AI systems and backend services regardless of the underlying framework.
How does the Agent Gateway handle observability and cost tracking?
The gateway provides deep visibility into the agent mesh. It tracks traffic management not just at the API level, but at the intent level. Teams get granular metrics on Model Context Protocol (MCP) usage, token consumption across LLM Models, and interaction costs with external tools. This intelligent routing and tracking capability allows teams to audit data sources and attribute costs accurately, solving a lot of problems related to "black box" agent behavior in production.
Can teams control and audit autonomous agents?
Absolutely. The Agent Gateway enforces RBAC, quotas, and guardrails at the agent level. All agent actions including model calls and tool invocations are logged and auditable, ensuring full traceability for security and compliance needs.
Is the Agent Gateway suitable for regulated or enterprise environments?
Yes. The Agent Gateway is designed for enterprise use cases and supports secure deployments in private VPCs, on-premise environments, and air-gapped setups. It integrates with existing compliance controls and provides centralized governance required for regulated industries.
How can I get started with the TrueFoundry Agent Gateway?
You can get started by signing in to TrueFoundry and deploying your first agent through the Agent Gateway. For teams with advanced or enterprise requirements, you can also book a demo to explore a customized deployment and governance strategy.
What does TrueFoundry do?
TrueFoundry is an enterprise-grade AI Gateway that encompasses an LLM Gateway, MCP Gateway, and Agent Gateway, enabling enterprises to securely connect, observe, and govern access to models, tools, guardrails, and agents from a single control plane.
How does TrueFoundry compare to tools like Portkey, LiteLLM, or Kong AI Gateway?
Those tools focus primarily on API management and routing. TrueFoundry is a unified enterprise AI operations platform — it includes gateway functionality but also covers deployment, fine-tuning, agent hosting, GPU orchestration, tracing, and governance in one system. It is also designed for on-prem, VPC, and air-gapped deployment.
Which compliance standards does TrueFoundry meet?
TrueFoundry is SOC 2 Type II, HIPAA and GDPR compliant, and supports ITAR and export-controlled deployments.