As enterprises scale their AI initiatives, a new challenge is emerging: uncontrolled costs, inconsistent governance, and fragmented integrations across models and providers. The Gartner® Market Guide for AI Gateways (October 2025) identifies this shift clearly, projecting that by 2028, 70% of software engineering teams building multimodel applications will use AI gateways to improve reliability and optimize costs, compared to 25% in 2025.

Gartner recognizes TrueFoundry as a representative vendor for our product, the TrueFoundry AI Gateway. Built as a unified control layer for AI workloads, it brings observability, cost control, and governance to enterprise-scale AI environments.

Read the full report 

Understanding the AI Gateway Layer

Gartner defines an AI Gateway as middleware that sits between applications and AI services or models, managing security, observability, and cost optimization.

At a technical level, an AI Gateway performs four foundational tasks:

1. Routing: Directs inference traffic to the most efficient model or provider according to latency, accuracy, or cost policies.
   
   
2. Security: Manages authentication, key rotation, and input or output guardrails from a central control point.
   
   
3. Cost Control: Tracks token usage, enforces quotas, and applies caching to reduce redundant model calls.
   
   
4. Observability: Provides metrics, traces, and performance analytics across every AI interaction.

This transforms the Gateway into a programmable policy and governance plane sitting between the application layer and the model ecosystem — similar to what Kubernetes did for containers.

At TrueFoundry, we view this layer as the AI Control Plane — the layer that brings together reliability, accountability, and operational excellence for multimodel and multi-agent systems.

Inside the Report’s Findings: Why AI Gateways Are Gaining Momentum

The Gartner® report identifies several forces accelerating AI Gateway adoption:

• Increasing traffic and token costs: Without centralized monitoring, organizations risk ungoverned AI usage. Gateways enable visibility, quota enforcement, and policy-driven spending.
  
  
• Security and compliance requirements: Gateways apply AI Trust, Risk, and Security Management (AI TRiSM) policies, ensuring safe prompts, data redaction, and regulated access.
  
  
• Expanding use cases: Initially designed for LLMs, Gateways are now expected to handle Model Context Protocol (MCP) and A2A (AI-to-AI) communication.
  
  
• Market immaturity: Capabilities vary widely between vendors. Gartner recommends choosing platforms with proven adaptability, innovation roadmaps, and strong integration capabilities.

In essence, Gateways are shifting from simple traffic routers to intelligent governance engines that define how enterprise AI operates.

‍

What CIOs and Platform Teams Should Evaluate

Based on the evaluation framework in the Market Guide for AI Gateways, enterprises should assess AI Gateway solutions on the following dimensions:

1. Integration with Existing Systems: Compatibility with existing API gateways or service meshes without extensive re-architecture.
   
   
2. Vendor Innovation Roadmap: Evidence of support for emerging standards such as MCP, A2A, and hybrid edge deployment.
   
   
3. Security and Compliance Strength: Ability to safeguard credentials, block unsafe inputs, and maintain audit trails.
   
   
4. Scalability and Latency Performance: Proven handling of high-throughput workloads while minimizing round-trip delays.
   
   
5. Pricing Alignment: Transparent models that map to token consumption and enterprise budgeting cycles.
   
   
6. Observability and Transparency: Built-in telemetry, cost analytics, and alerting for proactive governance.
   
   
7. Integration with AI Security Tools: Compatibility with TRiSM or AI posture-management platforms to extend enterprise security coverage.

These considerations reflect a fundamental change in how organizations evaluate AI infrastructure: not only by accuracy or performance but by reliability, safety, and cost predictability.

How Enterprises Deploy Gateways

As enterprises operationalize AI Gateways, several deployment patterns are emerging across the market—each balancing central control with distributed performance.

Aggregator: A single gateway aggregates providers behind it, enforcing global policy and cross-model orchestration.

Proxy: Fronts individual providers for fast rollout of authentication and logging without re-architecture.

Composite / Hybrid: Regional gateways feed a global control layer for low-latency access under central governance.

These reflect the architectural flexibility Gartner emphasizes—supporting multi-cloud, hybrid, and on-prem models while maintaining unified governance and observability.
TrueFoundry AI Gateway supports all three: defaulting to Aggregator mode, scaling into Composite topologies for global rollouts, and offering Proxy setups for selective control.

‍

How TrueFoundry AI Gateway Works

1. Unified Ingress and Routing Engine

The routing layer dynamically directs traffic based on performance, reliability, or cost criteria. It supports multiple AI providers such as OpenAI, Anthropic, Mistral, and Cohere, and automatically fails over when a model or endpoint becomes unstable.

2. Token-Aware Cost Optimization

TrueFoundry’s adaptive cost engine tracks token consumption by application, user, and model. Semantic caching reduces redundant LLM requests by up to 40 percent without sacrificing response quality, lowering spend and improving responsiveness.

3. Security and Guardrail Layer

Centralized key rotation eliminates credential sprawl. Input and output validation guardrails block unsafe prompts, redact sensitive data, and enforce compliance standards such as HIPAA and GDPR. Role-based and policy-based access control uses OAuth 2.x and OIDC for secure authentication.

4. Observability and Telemetry

Real-time tracing, latency analytics, and usage dashboards provide visibility into every AI interaction. Integrations with Grafana, Datadog, and OpenTelemetry allow unified monitoring across environments.

5. MCP Gateway Interoperability

TrueFoundry’s AI Gateway natively supports the Model Context Protocol, allowing AI agents to access enterprise systems through a secure, auditable control plane. This creates a seamless bridge between AI model governance and multi-agent orchestration.

Market Direction: The Shift from API to AI Management

Gartner notes a rapid convergence between API management and AI control systems. Traditional API vendors are adding AI-specific extensions, while AI platform providers like TrueFoundry deliver gateways that are natively optimized for token management, semantic caching, and latency analytics.

The total addressable market for AI Gateways is estimated at 50 to 100 million dollars in 2025, and it is expected to grow sharply as enterprises adopt multimodel and hybrid AI architectures. Gartner predicts increasing vendor consolidation, expansion into edge deployments, and deeper integration with AI observability tools.

Going Beyond the Gartner Checklist

The Gartner Market Guide lays out what enterprises should evaluate before adopting AI Gateways. To our, TrueFoundry AI Gateway not only meets those criteria—it extends them with multi-provider routing, agent-aware governance, and deep observability.

Here’s a snapshot of how we believe TrueFoundry aligns with and goes beyond the report’s recommended capabilities.

This architecture enables platform and DevOps teams to operate AI systems with the same rigor they apply to production software; measurable, secure, and compliant.

To see the full enterprise checklist—including advanced criteria like multi-region deployments and MCP integration, download the AI Gateway Evaluation Checklist.

Value You Can Prove in 90 Days

Month 1: Centralize keys, activate token dashboards, and audit credentials for shadow usage.
Outcome: コストの可視化と認証情報のセキュリティ。

2ヶ月目： IDプロバイダー（Okta、Azure AD）を接続し、クォータと予算を有効化し、ガバナンスポリシーを展開します。
成果： 予測可能な支出とポリシー適用範囲。

3ヶ月目： ルーティング、フェイルオーバー、セマンティックキャッシュを有効化し、アップタイムとコスト最適化を確保します。
成果： 高い回復性と完全な監査可能性。

経営層向けKPI： 支出の予測可能性 • プロバイダーの回復性 • コンプライアンスの追跡可能性

今後の展望

AIゲートウェイは、複雑なモデルエコシステムを管理するために必要なガバナンスと可視性を提供し、エンタープライズAIの基盤となりつつあります。Gartner 2025 Market Guideは、この変化を裏付けており、TrueFoundryのようなベンダーがこの不可欠なインフラ層を進化させていることを評価しています。

組織がマルチモデルおよびマルチエージェントAIシステムへと進化するにつれて、TrueFoundryは、ルーティング、セキュリティ、コスト管理が集約される基盤となるコントロールプレーンの構築に注力しています。

“AIイノベーションの次の段階は、いかにインテリジェントにガバナンスされるかにかかっています。AIゲートウェイは、そのガバナンスを可能にします。.”

について詳しくはこちら TrueFoundry AIゲートウェイ そして、あらゆるモデルや環境において、安全で、可観測性があり、費用対効果の高いAI運用をどのように実現するか。

ガートナー AIゲートウェイ市場ガイド、Andrew Humphreys 他、2025年10月13日

GARTNERは、Gartner, Inc.および/またはその米国および国際的な関連会社の登録商標およびサービスマークであり、本書では許可を得て使用されています。全ての権利は留保されています。ガートナーは、その調査出版物に掲載されているいかなるベンダー、製品、サービスも推奨するものではなく、テクノロジーユーザーに対し、最高の評価またはその他の指定を受けたベンダーのみを選択するよう助言するものではありません。ガートナーの調査出版物は、ガートナーの調査組織の意見で構成されており、事実の記述と解釈されるべきではありません。ガートナーは、本調査に関して、商品性または特定目的への適合性の保証を含む、明示的または黙示的な一切の保証を否認します。

この図は、Gartner, Inc.がより大規模な調査文書の一部として発行したものであり、文書全体との関連において評価されるべきです。ガートナーの文書は、TrueFoundryに請求すれば入手可能です。