> ## Documentation Index
> Fetch the complete documentation index at: https://www.truefoundry.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Federated MCP Gateway

> Deploy MCP Gateway across multiple clouds and on-premises with federated access, enabling cross-region tool invocation through a unified control plane.

Enterprise organizations often have MCP servers distributed across multiple cloud providers and on-premises data centers. These MCP servers may be accessible only from within their respective network environments due to security policies, compliance requirements, or network isolation. The TrueFoundry Federated MCP Gateway architecture enables seamless cross-cloud access to MCP servers while maintaining network isolation and security boundaries.

## The Challenge: Multi-Cloud MCP Server Access

Consider an organization with the following infrastructure:

* **AWS**: Internal analytics MCP server, customer data tools
* **Azure**: Microsoft 365 integration MCP server, Azure DevOps tools
* **GCP**: BigQuery MCP server, Google Workspace tools
* **On-premises**: Legacy systems MCP server, sensitive data tools

Each of these MCP servers is only accessible from within their respective cloud or network environment. An application running in AWS cannot directly reach an MCP server deployed in Azure due to network isolation.

**Common requirements:**

* Applications in AWS need to access MCP servers in Azure, GCP, and on-premises
* Applications in Azure need to access MCP servers in AWS, GCP, and on-premises
* Network isolation must be maintained—direct cross-cloud connections are not allowed
* A single control plane should manage all MCP server registrations and access policies
* Users should have a unified experience regardless of where MCP servers are deployed

***

## Federated Gateway Architecture

The TrueFoundry Federated MCP Gateway solves this challenge by deploying gateway planes in each cloud environment, with all gateway planes connected to a central control plane.

### Key Components

| Component                       | Description                                                                                                                                                             |
| ------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Central Control Plane**       | Single control plane deployed in a central region. Manages all MCP server registrations, access policies, and configuration. Syncs configuration to all gateway planes. |
| **Gateway Planes**              | One gateway plane deployed in each cloud/environment (AWS, Azure, GCP, on-premises). Each gateway plane can only access MCP servers within its own environment.         |
| **Inter-Gateway Communication** | All gateway planes can communicate with each other and with the control plane. This enables request proxying across environments.                                       |
| **MCP Servers**                 | MCP servers deployed within each cloud environment. Only accessible from the gateway plane in the same environment.                                                     |

### Network Topology

<img src="https://mintcdn.com/truefoundry/ohW9qMgZsptlEVKe/images/docs/ai-gateway/federated-mcp-gateway.png?fit=max&auto=format&n=ohW9qMgZsptlEVKe&q=85&s=70027606c66f5b640366c1e5815e41c3" alt="Federated MCP Gateway Architecture" width="7632" height="5160" data-path="images/docs/ai-gateway/federated-mcp-gateway.png" />

<Note>
  Each gateway plane has network access only to MCP servers within its own environment. Cross-environment access is achieved through gateway-to-gateway proxying.
</Note>

***

## How Federated Routing Works

When an MCP server is registered in the TrueFoundry control plane, you specify a **proxy URL** that indicates which gateway plane can directly access that MCP server. The control plane syncs this information to all gateway planes, enabling intelligent request routing.

### Proxy URL Configuration

When registering an MCP server, you configure:

| Field              | Description                                           | Example                              |
| ------------------ | ----------------------------------------------------- | ------------------------------------ |
| **MCP Server URL** | The internal URL of the MCP server                    | `http://analytics-mcp.internal:8080` |
| **Proxy URL**      | The gateway plane URL that can access this MCP server | `https://aws-gw.example.com`         |

For example:

* `mcp-analytics` (deployed in AWS) → Proxy URL: `https://aws-gw.example.com`
* `mcp-m365` (deployed in Azure) → Proxy URL: `https://azure-gw.example.com`
* `mcp-bigquery` (deployed in GCP) → Proxy URL: `https://gcp-gw.example.com`

### Request Flow: Cross-Cloud Access

Let's trace a request from an application in AWS accessing an MCP server deployed in Azure.

**Scenario:** An AI agent running in AWS needs to call a tool on `mcp-m365` (deployed in Azure).

```mermaid theme={"dark"}
sequenceDiagram
    participant App as Application (AWS)
    participant AWSGw as AWS Gateway<br/>aws-gw.example.com
    participant AzureGw as Azure Gateway<br/>azure-gw.example.com
    participant MCP as mcp-m365<br/>(Azure)

    App->>AWSGw: 1. Tool call request
    Note over AWSGw: 2. Lookup mcp-m365<br/>Proxy URL: azure-gw.example.com
    AWSGw->>AzureGw: 3. Proxy request to Azure Gateway
    Note over AzureGw: 4. Lookup mcp-m365<br/>Proxy URL matches self
    AzureGw->>MCP: 5. Forward to MCP server
    MCP-->>AzureGw: 6. Response
    AzureGw-->>AWSGw: 7. Relay response
    AWSGw-->>App: 8. Return result
```

<Steps>
  <Step title="Application sends request to local gateway">
    The application in AWS sends a tool call request to its local gateway plane (`aws-gw.example.com`). The request includes the MCP server identifier (`mcp-m365`) and authentication credentials.
  </Step>

  <Step title="AWS Gateway looks up MCP server configuration">
    The AWS Gateway retrieves the configuration for `mcp-m365` from its synced configuration. It sees that the proxy URL is `azure-gw.example.com`, which is different from itself.
  </Step>

  <Step title="AWS Gateway proxies request to Azure Gateway">
    Since the proxy URL doesn't match its own URL, the AWS Gateway forwards the request to the Azure Gateway (`azure-gw.example.com`). The request includes all necessary authentication and context.
  </Step>

  <Step title="Azure Gateway receives and evaluates request">
    The Azure Gateway receives the proxied request and looks up the configuration for `mcp-m365`. It sees the proxy URL is `azure-gw.example.com`, which matches its own URL.
  </Step>

  <Step title="Azure Gateway forwards to MCP server">
    Since the proxy URL matches its own URL, the Azure Gateway knows it should handle this request directly. It forwards the request to the actual MCP server (`mcp-m365`) running in Azure.
  </Step>

  <Step title="Response flows back through the chain">
    The MCP server processes the request and returns the response. The response flows back through Azure Gateway → AWS Gateway → Application.
  </Step>
</Steps>

### Proxy URL Matching Logic

Each gateway plane uses the following logic when handling a request:

```
IF (mcp_server.proxy_url == gateway.own_url) THEN
    # This gateway should handle the request directly
    Forward request to mcp_server.internal_url
ELSE
    # This gateway should proxy to another gateway
    Forward request to mcp_server.proxy_url
END IF
```

<Info>
  The proxy URL matching is what enables the "hop" behavior. A gateway receiving a proxied request will check if it's the final destination before forwarding to the MCP server.
</Info>

***

## Configuration Guide

### Step 1: Deploy Gateway Planes in Each Environment

Deploy a TrueFoundry Gateway Plane in each cloud environment where you have MCP servers.

<Tabs>
  <Tab title="AWS">
    Deploy the gateway plane in your AWS environment with a publicly accessible URL:

    ```yaml theme={"dark"}
    # Gateway configuration
    gateway:
      url: https://aws-gw.example.com
      controlPlane:
        url: https://control-plane.example.com
        natsUrl: nats://control-plane.example.com:4222
    ```

    Ensure the gateway has network access to:

    * All MCP servers within AWS VPC
    * The central control plane
    * Other gateway planes (Azure, GCP, on-premises)
  </Tab>

  <Tab title="Azure">
    Deploy the gateway plane in your Azure environment:

    ```yaml theme={"dark"}
    # Gateway configuration
    gateway:
      url: https://azure-gw.example.com
      controlPlane:
        url: https://control-plane.example.com
        natsUrl: nats://control-plane.example.com:4222
    ```

    Ensure the gateway has network access to:

    * All MCP servers within Azure VNet
    * The central control plane
    * Other gateway planes (AWS, GCP, on-premises)
  </Tab>

  <Tab title="GCP">
    Deploy the gateway plane in your GCP environment:

    ```yaml theme={"dark"}
    # Gateway configuration
    gateway:
      url: https://gcp-gw.example.com
      controlPlane:
        url: https://control-plane.example.com
        natsUrl: nats://control-plane.example.com:4222
    ```

    Ensure the gateway has network access to:

    * All MCP servers within GCP VPC
    * The central control plane
    * Other gateway planes (AWS, Azure, on-premises)
  </Tab>

  <Tab title="On-premises">
    Deploy the gateway plane in your on-premises data center:

    ```yaml theme={"dark"}
    # Gateway configuration
    gateway:
      url: https://onprem-gw.example.com
      controlPlane:
        url: https://control-plane.example.com
        natsUrl: nats://control-plane.example.com:4222
    ```

    Ensure the gateway has network access to:

    * All MCP servers within on-premises network
    * The central control plane (may require VPN or ExpressRoute)
    * Other gateway planes (may require VPN or dedicated connectivity)
  </Tab>
</Tabs>

<Note>
  For detailed gateway plane deployment instructions, see [Gateway Plane Architecture](/docs/platform/gateway-plane-architecture) and [Deploy Gateway Plane](/docs/platform/deploy-gateway-plane).
</Note>

### Step 2: Configure Inter-Gateway Connectivity

All gateway planes must be able to communicate with each other. Configure your network to allow:

| Source          | Destination     | Port | Protocol |
| --------------- | --------------- | ---- | -------- |
| AWS Gateway     | Azure Gateway   | 443  | HTTPS    |
| AWS Gateway     | GCP Gateway     | 443  | HTTPS    |
| AWS Gateway     | On-prem Gateway | 443  | HTTPS    |
| Azure Gateway   | AWS Gateway     | 443  | HTTPS    |
| Azure Gateway   | GCP Gateway     | 443  | HTTPS    |
| Azure Gateway   | On-prem Gateway | 443  | HTTPS    |
| GCP Gateway     | AWS Gateway     | 443  | HTTPS    |
| GCP Gateway     | Azure Gateway   | 443  | HTTPS    |
| GCP Gateway     | On-prem Gateway | 443  | HTTPS    |
| On-prem Gateway | AWS Gateway     | 443  | HTTPS    |
| On-prem Gateway | Azure Gateway   | 443  | HTTPS    |
| On-prem Gateway | GCP Gateway     | 443  | HTTPS    |

<Warning>
  Ensure all gateway-to-gateway connections use TLS and are authenticated. The gateways use mutual authentication to verify each other's identity.
</Warning>

### Step 3: Register MCP Servers with Proxy URLs

When registering MCP servers in the TrueFoundry control plane, specify the proxy URL for each server.

<Steps>
  <Step title="Navigate to MCP Registry">
    Go to **AI Gateway > MCP Servers** in the TrueFoundry UI.
  </Step>

  <Step title="Add a new MCP server">
    Click **Add MCP Server** and fill in the details:

    | Field          | Value                                 |
    | -------------- | ------------------------------------- |
    | Name           | `mcp-m365`                            |
    | Server URL     | `http://m365-mcp.azure-internal:8080` |
    | Proxy URL      | `https://azure-gw.example.com`        |
    | Authentication | Configure as needed                   |
  </Step>

  <Step title="Repeat for all MCP servers">
    Register all MCP servers across all environments, ensuring each has the correct proxy URL pointing to its local gateway plane.
  </Step>
</Steps>

**Example registrations:**

| MCP Server      | Internal URL                         | Proxy URL                       |
| --------------- | ------------------------------------ | ------------------------------- |
| `mcp-analytics` | `http://analytics.aws-internal:8080` | `https://aws-gw.example.com`    |
| `mcp-m365`      | `http://m365.azure-internal:8080`    | `https://azure-gw.example.com`  |
| `mcp-devops`    | `http://devops.azure-internal:8080`  | `https://azure-gw.example.com`  |
| `mcp-bigquery`  | `http://bigquery.gcp-internal:8080`  | `https://gcp-gw.example.com`    |
| `mcp-legacy`    | `http://legacy.onprem-internal:8080` | `https://onprem-gw.example.com` |

### Step 4: Configure Applications to Use Local Gateway

Applications should always connect to their local gateway plane. The gateway handles all cross-cloud routing automatically.

```python theme={"dark"}
from fastmcp import Client
from fastmcp.client.transports import StreamableHttpTransport

# Application running in AWS connects to AWS Gateway
transport = StreamableHttpTransport(
    url="https://aws-gw.example.com/api/llm/mcp/mcp-m365/server",  # MCP server in Azure
    headers={"Authorization": "Bearer <your-tfy-api-key>"}
)

async with Client(transport) as client:
    # This request is automatically routed: AWS Gateway → Azure Gateway → mcp-m365
    result = await client.call_tool("get_calendar_events", {"date": "2026-02-06"})
    print(result)
```

<Tip>
  Applications don't need to know where MCP servers are deployed. They always connect to their local gateway, and the gateway handles the routing transparently.
</Tip>

***

## Authentication in Federated Setup

All standard authentication methods work in the federated setup. The authentication token is passed through the entire proxy chain.

### Token Flow in Proxied Requests

```mermaid theme={"dark"}
sequenceDiagram
    participant App as Application
    participant LocalGw as Local Gateway
    participant RemoteGw as Remote Gateway
    participant MCP as MCP Server

    App->>LocalGw: Request + Auth Token
    Note over LocalGw: Validate token<br/>Check access control
    LocalGw->>RemoteGw: Proxied Request + Auth Token
    Note over RemoteGw: Validate token<br/>Check access control
    RemoteGw->>MCP: Request + Outbound Auth
    MCP-->>RemoteGw: Response
    RemoteGw-->>LocalGw: Response
    LocalGw-->>App: Response
```

<AccordionGroup>
  <Accordion title="TrueFoundry API Key / PAT">
    The Personal Access Token is validated at each gateway in the chain. Both gateways verify the token and check access control policies before forwarding the request.
  </Accordion>

  <Accordion title="Identity Provider Token">
    JWTs from a configured [Identity Provider](/docs/platform/identity-providers) are validated at each gateway using the provider's JWKS, issuer, and allowed audiences. The token's resolved TrueFoundry identity (user or virtual account) is used for access control.
  </Accordion>

  <Accordion title="OAuth2 Outbound Authentication">
    For MCP servers requiring OAuth2 (like Slack or GitHub), the remote gateway retrieves the user's OAuth token from the control plane and uses it to authenticate with the MCP server.
  </Accordion>
</AccordionGroup>

<Note>
  Access control policies are enforced at both the local and remote gateways. A user must have permission to access the MCP server at both gateways for the request to succeed.
</Note>

***

## Observability and Monitoring

The federated setup provides full observability across all gateway planes.

### Request Tracing

Each request includes trace context that flows through the entire proxy chain. You can view:

* End-to-end latency (including inter-gateway hops)
* Which gateway planes handled the request
* Time spent at each hop
* MCP server response time

### Metrics

All gateway planes report metrics to the central control plane:

| Metric                          | Description                                     |
| ------------------------------- | ----------------------------------------------- |
| `mcp_requests_total`            | Total MCP requests by server, user, and gateway |
| `mcp_proxy_requests_total`      | Requests proxied to other gateways              |
| `mcp_proxy_latency_ms`          | Latency added by inter-gateway proxying         |
| `mcp_gateway_to_gateway_errors` | Errors in gateway-to-gateway communication      |

### Audit Logging

All requests are logged with complete audit trail:

```json theme={"dark"}
{
  "timestamp": "2026-02-06T10:30:00Z",
  "user": "user@example.com",
  "mcp_server": "mcp-m365",
  "tool": "get_calendar_events",
  "entry_gateway": "aws-gw.example.com",
  "handling_gateway": "azure-gw.example.com",
  "proxy_hops": 1,
  "total_latency_ms": 245,
  "status": "success"
}
```

***

## Best Practices

<AccordionGroup>
  <Accordion title="Gateway Placement">
    * Deploy gateway planes as close as possible to the MCP servers they serve
    * Use dedicated gateway instances for high-traffic MCP servers
    * Consider deploying multiple gateway replicas for high availability
  </Accordion>

  <Accordion title="Network Configuration">
    * Use private connectivity (VPC peering, ExpressRoute, Cloud Interconnect) between clouds when possible
    * Ensure low-latency connections between gateway planes
    * Configure appropriate timeouts for inter-gateway communication
  </Accordion>

  <Accordion title="Security">
    * Use mutual TLS for all gateway-to-gateway communication
    * Implement network segmentation—MCP servers should only be accessible from their local gateway
    * Regularly rotate certificates and credentials
  </Accordion>

  <Accordion title="Monitoring">
    * Set up alerts for inter-gateway communication failures
    * Monitor proxy latency and set thresholds for acceptable performance
    * Track cross-cloud traffic for cost optimization
  </Accordion>
</AccordionGroup>

***

## FAQ

<AccordionGroup>
  <Accordion title="What happens if a gateway plane is unavailable?">
    If a remote gateway is unavailable, requests to MCP servers behind that gateway will fail. The local gateway will return an error indicating the remote gateway is unreachable. We recommend deploying multiple gateway replicas and using load balancers for high availability.
  </Accordion>

  <Accordion title="Does proxying add significant latency?">
    Inter-gateway proxying adds latency based on the network distance between gateways. For same-region gateways, expect 5-20ms additional latency. For cross-region (e.g., US to Europe), expect 50-150ms additional latency. The gateway is optimized to minimize overhead.
  </Accordion>

  <Accordion title="Can I have multiple gateways in the same cloud?">
    Yes, you can deploy multiple gateway planes in the same cloud for different purposes (e.g., production vs. staging, or different VPCs). Each gateway has its own URL and can serve as a proxy destination for specific MCP servers.
  </Accordion>

  <Accordion title="How does configuration sync work across gateways?">
    All gateway planes subscribe to the central control plane via NATS. When you register or update an MCP server (including its proxy URL), the configuration is synced to all gateway planes within seconds. See [Gateway Plane Architecture](/docs/platform/gateway-plane-architecture) for details.
  </Accordion>

  <Accordion title="Can applications access multiple MCP servers in different clouds in a single session?">
    Yes. An application connects to its local gateway once and can access any MCP server regardless of where it's deployed. The gateway handles all routing transparently. Each tool call is routed to the appropriate gateway based on the MCP server's proxy URL.
  </Accordion>

  <Accordion title="How are errors handled in the proxy chain?">
    Errors are propagated back through the chain with full context. If the remote MCP server returns an error, it's passed back to the application. If there's a network error between gateways, the application receives a gateway error with details about which hop failed.
  </Accordion>
</AccordionGroup>