Federated MCP Gateway - TrueFoundry Docs

Enterprise organizations often have MCP servers distributed across multiple cloud providers and on-premises data centers. These MCP servers may be accessible only from within their respective network environments due to security policies, compliance requirements, or network isolation. The TrueFoundry Federated MCP Gateway architecture enables seamless cross-cloud access to MCP servers while maintaining network isolation and security boundaries.

The Challenge: Multi-Cloud MCP Server Access

Consider an organization with the following infrastructure:

AWS: Internal analytics MCP server, customer data tools
Azure: Microsoft 365 integration MCP server, Azure DevOps tools
GCP: BigQuery MCP server, Google Workspace tools
On-premises: Legacy systems MCP server, sensitive data tools

Each of these MCP servers is only accessible from within their respective cloud or network environment. An application running in AWS cannot directly reach an MCP server deployed in Azure due to network isolation. Common requirements:

Applications in AWS need to access MCP servers in Azure, GCP, and on-premises
Applications in Azure need to access MCP servers in AWS, GCP, and on-premises
Network isolation must be maintained—direct cross-cloud connections are not allowed
A single control plane should manage all MCP server registrations and access policies
Users should have a unified experience regardless of where MCP servers are deployed

Federated Gateway Architecture

The TrueFoundry Federated MCP Gateway solves this challenge by deploying gateway planes in each cloud environment, with all gateway planes connected to a central control plane.

Key Components

Component	Description
Central Control Plane	Single control plane deployed in a central region. Manages all MCP server registrations, access policies, and configuration. Syncs configuration to all gateway planes.
Gateway Planes	One gateway plane deployed in each cloud/environment (AWS, Azure, GCP, on-premises). Each gateway plane can only access MCP servers within its own environment.
Inter-Gateway Communication	All gateway planes can communicate with each other and with the control plane. This enables request proxying across environments.
MCP Servers	MCP servers deployed within each cloud environment. Only accessible from the gateway plane in the same environment.

Network Topology

Each gateway plane has network access only to MCP servers within its own environment. Cross-environment access is achieved through gateway-to-gateway proxying.

How Federated Routing Works

When an MCP server is registered in the TrueFoundry control plane, you specify a proxy URL that indicates which gateway plane can directly access that MCP server. The control plane syncs this information to all gateway planes, enabling intelligent request routing.

Proxy URL Configuration

When registering an MCP server, you configure:

Field	Description	Example
MCP Server URL	The internal URL of the MCP server	`http://analytics-mcp.internal:8080`
Proxy URL	The gateway plane URL that can access this MCP server	`https://aws-gw.example.com`

For example:

mcp-analytics (deployed in AWS) → Proxy URL: https://aws-gw.example.com
mcp-m365 (deployed in Azure) → Proxy URL: https://azure-gw.example.com
mcp-bigquery (deployed in GCP) → Proxy URL: https://gcp-gw.example.com

Request Flow: Cross-Cloud Access

Let’s trace a request from an application in AWS accessing an MCP server deployed in Azure. Scenario: An AI agent running in AWS needs to call a tool on mcp-m365 (deployed in Azure).

Application sends request to local gateway

The application in AWS sends a tool call request to its local gateway plane (aws-gw.example.com). The request includes the MCP server identifier (mcp-m365) and authentication credentials.

AWS Gateway looks up MCP server configuration

The AWS Gateway retrieves the configuration for mcp-m365 from its synced configuration. It sees that the proxy URL is azure-gw.example.com, which is different from itself.

AWS Gateway proxies request to Azure Gateway

Since the proxy URL doesn’t match its own URL, the AWS Gateway forwards the request to the Azure Gateway (azure-gw.example.com). The request includes all necessary authentication and context.

Azure Gateway receives and evaluates request

The Azure Gateway receives the proxied request and looks up the configuration for mcp-m365. It sees the proxy URL is azure-gw.example.com, which matches its own URL.

Azure Gateway forwards to MCP server

Since the proxy URL matches its own URL, the Azure Gateway knows it should handle this request directly. It forwards the request to the actual MCP server (mcp-m365) running in Azure.

Response flows back through the chain

The MCP server processes the request and returns the response. The response flows back through Azure Gateway → AWS Gateway → Application.

Proxy URL Matching Logic

Each gateway plane uses the following logic when handling a request:

IF (mcp_server.proxy_url == gateway.own_url) THEN
    # This gateway should handle the request directly
    Forward request to mcp_server.internal_url
ELSE
    # This gateway should proxy to another gateway
    Forward request to mcp_server.proxy_url
END IF

The proxy URL matching is what enables the “hop” behavior. A gateway receiving a proxied request will check if it’s the final destination before forwarding to the MCP server.

Configuration Guide

Step 1: Deploy Gateway Planes in Each Environment

Deploy a TrueFoundry Gateway Plane in each cloud environment where you have MCP servers.

AWS
Azure
GCP
On-premises

Deploy the gateway plane in your AWS environment with a publicly accessible URL:

# Gateway configuration
gateway:
  url: https://aws-gw.example.com
  controlPlane:
    url: https://control-plane.example.com
    natsUrl: nats://control-plane.example.com:4222

Ensure the gateway has network access to:

All MCP servers within AWS VPC
The central control plane
Other gateway planes (Azure, GCP, on-premises)

Deploy the gateway plane in your Azure environment:

# Gateway configuration
gateway:
  url: https://azure-gw.example.com
  controlPlane:
    url: https://control-plane.example.com
    natsUrl: nats://control-plane.example.com:4222

Ensure the gateway has network access to:

All MCP servers within Azure VNet
The central control plane
Other gateway planes (AWS, GCP, on-premises)

Deploy the gateway plane in your GCP environment:

# Gateway configuration
gateway:
  url: https://gcp-gw.example.com
  controlPlane:
    url: https://control-plane.example.com
    natsUrl: nats://control-plane.example.com:4222

Ensure the gateway has network access to:

All MCP servers within GCP VPC
The central control plane
Other gateway planes (AWS, Azure, on-premises)

Deploy the gateway plane in your on-premises data center:

# Gateway configuration
gateway:
  url: https://onprem-gw.example.com
  controlPlane:
    url: https://control-plane.example.com
    natsUrl: nats://control-plane.example.com:4222

Ensure the gateway has network access to:

All MCP servers within on-premises network
The central control plane (may require VPN or ExpressRoute)
Other gateway planes (may require VPN or dedicated connectivity)

For detailed gateway plane deployment instructions, see Gateway Plane Architecture and Deploy Gateway Plane.

Step 2: Configure Inter-Gateway Connectivity

All gateway planes must be able to communicate with each other. Configure your network to allow:

Source	Destination	Port	Protocol
AWS Gateway	Azure Gateway	443	HTTPS
AWS Gateway	GCP Gateway	443	HTTPS
AWS Gateway	On-prem Gateway	443	HTTPS
Azure Gateway	AWS Gateway	443	HTTPS
Azure Gateway	GCP Gateway	443	HTTPS
Azure Gateway	On-prem Gateway	443	HTTPS
GCP Gateway	AWS Gateway	443	HTTPS
GCP Gateway	Azure Gateway	443	HTTPS
GCP Gateway	On-prem Gateway	443	HTTPS
On-prem Gateway	AWS Gateway	443	HTTPS
On-prem Gateway	Azure Gateway	443	HTTPS
On-prem Gateway	GCP Gateway	443	HTTPS

Ensure all gateway-to-gateway connections use TLS and are authenticated. The gateways use mutual authentication to verify each other’s identity.

Step 3: Register MCP Servers with Proxy URLs

When registering MCP servers in the TrueFoundry control plane, specify the proxy URL for each server.

Navigate to MCP Registry

Go to AI Gateway > MCP Servers in the TrueFoundry UI.

Add a new MCP server

Click Add MCP Server and fill in the details:

Field	Value
Name	`mcp-m365`
Server URL	`http://m365-mcp.azure-internal:8080`
Proxy URL	`https://azure-gw.example.com`
Authentication	Configure as needed

Repeat for all MCP servers

Register all MCP servers across all environments, ensuring each has the correct proxy URL pointing to its local gateway plane.

Example registrations:

MCP Server	Internal URL	Proxy URL
`mcp-analytics`	`http://analytics.aws-internal:8080`	`https://aws-gw.example.com`
`mcp-m365`	`http://m365.azure-internal:8080`	`https://azure-gw.example.com`
`mcp-devops`	`http://devops.azure-internal:8080`	`https://azure-gw.example.com`
`mcp-bigquery`	`http://bigquery.gcp-internal:8080`	`https://gcp-gw.example.com`
`mcp-legacy`	`http://legacy.onprem-internal:8080`	`https://onprem-gw.example.com`

Step 4: Configure Applications to Use Local Gateway

Applications should always connect to their local gateway plane. The gateway handles all cross-cloud routing automatically.

from fastmcp import Client
from fastmcp.client.transports import StreamableHttpTransport

# Application running in AWS connects to AWS Gateway
transport = StreamableHttpTransport(
    url="https://aws-gw.example.com/api/llm/mcp/mcp-m365/server",  # MCP server in Azure
    headers={"Authorization": "Bearer <your-tfy-api-key>"}
)

async with Client(transport) as client:
    # This request is automatically routed: AWS Gateway → Azure Gateway → mcp-m365
    result = await client.call_tool("get_calendar_events", {"date": "2026-02-06"})
    print(result)

Applications don’t need to know where MCP servers are deployed. They always connect to their local gateway, and the gateway handles the routing transparently.

Authentication in Federated Setup

All standard authentication methods work in the federated setup. The authentication token is passed through the entire proxy chain.

Token Flow in Proxied Requests

TrueFoundry API Key / PAT

The Personal Access Token is validated at each gateway in the chain. Both gateways verify the token and check access control policies before forwarding the request.

External IdP Token

External IdP tokens are validated at each gateway using the configured SSO settings. The token claims are used to determine user identity and access permissions.

OAuth2 Outbound Authentication

For MCP servers requiring OAuth2 (like Slack or GitHub), the remote gateway retrieves the user’s OAuth token from the control plane and uses it to authenticate with the MCP server.

Access control policies are enforced at both the local and remote gateways. A user must have permission to access the MCP server at both gateways for the request to succeed.

Observability and Monitoring

The federated setup provides full observability across all gateway planes.

Request Tracing

Each request includes trace context that flows through the entire proxy chain. You can view:

End-to-end latency (including inter-gateway hops)
Which gateway planes handled the request
Time spent at each hop
MCP server response time

Metrics

All gateway planes report metrics to the central control plane:

Metric	Description
`mcp_requests_total`	Total MCP requests by server, user, and gateway
`mcp_proxy_requests_total`	Requests proxied to other gateways
`mcp_proxy_latency_ms`	Latency added by inter-gateway proxying
`mcp_gateway_to_gateway_errors`	Errors in gateway-to-gateway communication

Audit Logging

All requests are logged with complete audit trail:

{
  "timestamp": "2026-02-06T10:30:00Z",
  "user": "user@example.com",
  "mcp_server": "mcp-m365",
  "tool": "get_calendar_events",
  "entry_gateway": "aws-gw.example.com",
  "handling_gateway": "azure-gw.example.com",
  "proxy_hops": 1,
  "total_latency_ms": 245,
  "status": "success"
}

Best Practices

Gateway Placement

Deploy gateway planes as close as possible to the MCP servers they serve
Use dedicated gateway instances for high-traffic MCP servers
Consider deploying multiple gateway replicas for high availability

Network Configuration

Use private connectivity (VPC peering, ExpressRoute, Cloud Interconnect) between clouds when possible
Ensure low-latency connections between gateway planes
Configure appropriate timeouts for inter-gateway communication

Security

Use mutual TLS for all gateway-to-gateway communication
Implement network segmentation—MCP servers should only be accessible from their local gateway
Regularly rotate certificates and credentials

Monitoring

Set up alerts for inter-gateway communication failures
Monitor proxy latency and set thresholds for acceptable performance
Track cross-cloud traffic for cost optimization

FAQ

What happens if a gateway plane is unavailable?

If a remote gateway is unavailable, requests to MCP servers behind that gateway will fail. The local gateway will return an error indicating the remote gateway is unreachable. We recommend deploying multiple gateway replicas and using load balancers for high availability.

Does proxying add significant latency?

Inter-gateway proxying adds latency based on the network distance between gateways. For same-region gateways, expect 5-20ms additional latency. For cross-region (e.g., US to Europe), expect 50-150ms additional latency. The gateway is optimized to minimize overhead.

Can I have multiple gateways in the same cloud?

Yes, you can deploy multiple gateway planes in the same cloud for different purposes (e.g., production vs. staging, or different VPCs). Each gateway has its own URL and can serve as a proxy destination for specific MCP servers.

How does configuration sync work across gateways?

All gateway planes subscribe to the central control plane via NATS. When you register or update an MCP server (including its proxy URL), the configuration is synced to all gateway planes within seconds. See Gateway Plane Architecture for details.

Can applications access multiple MCP servers in different clouds in a single session?

Yes. An application connects to its local gateway once and can access any MCP server regardless of where it’s deployed. The gateway handles all routing transparently. Each tool call is routed to the appropriate gateway based on the MCP server’s proxy URL.

How are errors handled in the proxy chain?

Errors are propagated back through the chain with full context. If the remote MCP server returns an error, it’s passed back to the application. If there’s a network error between gateways, the application receives a gateway error with details about which hop failed.

Get Started

LLM Gateway

MCP Registry and Gateway

Agent Hub

Guardrails and Security

Prompt Management

Observability

Deployment

Admin Guide

Chat

Agent

Embeddings

Rerank

Responses

Image

Audio

Batch

Files

Moderations

Models

​The Challenge: Multi-Cloud MCP Server Access

​Federated Gateway Architecture

​Key Components

​Network Topology

​How Federated Routing Works

​Proxy URL Configuration

​Request Flow: Cross-Cloud Access

​Proxy URL Matching Logic

​Configuration Guide

​Step 1: Deploy Gateway Planes in Each Environment

​Step 2: Configure Inter-Gateway Connectivity

​Step 3: Register MCP Servers with Proxy URLs

​Step 4: Configure Applications to Use Local Gateway

​Authentication in Federated Setup

​Token Flow in Proxied Requests

​Observability and Monitoring

​Request Tracing

​Metrics

​Audit Logging

​Best Practices

​FAQ

The Challenge: Multi-Cloud MCP Server Access

Federated Gateway Architecture

Key Components

Network Topology

How Federated Routing Works

Proxy URL Configuration

Request Flow: Cross-Cloud Access

Proxy URL Matching Logic

Configuration Guide

Step 1: Deploy Gateway Planes in Each Environment

Step 2: Configure Inter-Gateway Connectivity

Step 3: Register MCP Servers with Proxy URLs

Step 4: Configure Applications to Use Local Gateway

Authentication in Federated Setup

Token Flow in Proxied Requests

Observability and Monitoring

Request Tracing

Metrics

Audit Logging

Best Practices

FAQ