EU-Hosted vs Self-Hosted AI: Who Runs Your Control Plane?

In June 2025, a Microsoft France executive told a French Senate inquiry, under oath, that he could not guarantee French public-sector data would never be passed to US authorities without French approval. His words were "No, I cannot guarantee it." He added that it had never happened.

That exchange is why "hosted in the EU" no longer settles an AI procurement conversation. Every vendor selling into Europe can say it. Far fewer can say who operates the system your prompts pass through, who holds the keys and who could be compelled to hand anything over.

EU hosting tells you where your data sits. Self-hosting tells you who controls it. For an AI gateway, the layer every model call passes through, that difference decides most of a vendor questionnaire.

In this blog, you will learn what separates the control plane from the data plane, where your data is processed during a single AI call, whether EU hosting protects you from the US CLOUD Act, what DORA changes for banks and insurers, and how to evaluate a sovereign AI gateway.

Control Plane vs Data Plane: The Distinction That Decides Everything

Every AI gateway has two parts.

The data plane is where requests flow. A prompt arrives, the gateway authenticates the caller, applies policy, forwards the call to a model and returns the response. It carries the content, so it is the most sensitive part of your AI stack.

The control plane is where the rules live: which teams can call which models, which guardrails apply, how budgets and rate limits are set. It holds configuration, identities and policy, and pushes them to the data plane.

That produces three common deployment shapes:

1. EU-hosted SaaS. The vendor runs both planes in a European data center. Your prompts, logs and policies sit on its infrastructure.
2. Hybrid. The gateway runs in your own cloud account or VPC, so LLM traffic stays on your infrastructure. The vendor hosts the control plane that holds configuration and policy.
3. Self-hosted. Both planes run in your environment. The vendor ships the software and you operate it.

Your data can be in Europe in all three. Only in the third does every part of the path run on infrastructure you control.

Where Is My Data Processed? Every Hop in an AI Gateway Call

Ask a vendor where your data is processed and you will usually hear one answer: a region. A single gateway call touches several places, and each has its own answer.

1. Authentication. Who is calling, and what gets recorded about them.
2. Input guardrails. PII detection and prompt-injection checks, which some gateways hand to a separate service.
3. Semantic cache. A stored copy of prompts and responses.
4. Model routing. The hop a region setting usually controls.
5. Output guardrails. The same question as step two, on the way back.
6. Logs and traces. Full prompts and completions are often written here for debugging, which can make this the biggest store of sensitive AI data you own.
7. Metering. Usage and cost data sent on for billing and analytics.

On an EU-hosted gateway, most of these steps run on the vendor's infrastructure. On a self-hosted gateway, they run on yours.

One caveat applies to every option: a gateway does not change where model routing sends a request. Route a prompt to a US-hosted model and it goes to the US. What you gain is the ability to enforce the rule yourself, such as EU-region models only for certain teams, and to check the logs afterwards. TrueFoundry evaluates residency constraints before routing and only selects model endpoints eligible for the workload's allowed region.

Does EU Hosting Protect You From the US CLOUD Act?

On its own, no.

The CLOUD Act, signed in 2018, requires providers within US jurisdiction to disclose data in their "possession, custody, or control", regardless of whether it is stored inside or outside the United States. If a US company operates your gateway, a Frankfurt data center changes where the data sits. It does not change which legal demands can reach the company running it.

Sovereign clouds run by European operators narrow that exposure. Whether they remove it entirely is a question for your legal team.

The structural argument is simpler. A provider can only be compelled to produce what it holds or controls. If your gateway, logs and keys run on your infrastructure and the vendor has no standing access to them, there is far less for anyone to compel. Far less is not nothing: a vendor still receives whatever its software sends back, which is why the checklist below asks exactly what leaves your environment.

Sovereign AI for European Banks: What DORA Changes

Since 17 January 2025, the Digital Operational Resilience Act (DORA, Regulation (EU) 2022/2554) has applied to EU banks, insurers, investment firms, payment institutions and other financial entities. Its third-party rules require a register of every ICT arrangement, specific contract clauses, a concentration-risk assessment and tested exit strategies for providers behind critical functions. The financial entity stays responsible for the risk, whoever supplies the service.

An AI gateway sits in front of every model call the business makes. If a vendor operates it, one external provider sits in the path of all AI traffic and holds the logs a supervisor may ask to see. It belongs on the register and in the concentration-risk assessment.

Self-hosting does not take the vendor off the register, because you still license its software. It changes what the exit plan has to cover. Logs and configuration already sit in your own estate, so leaving the vendor becomes a software migration rather than a data extraction.

What Is a Sovereign AI Gateway? A Buyer's Checklist

A sovereign AI gateway is one where you operate the request path and can show exactly what leaves your environment. Use these criteria to test any vendor, including us.

How TrueFoundry Handles Each Deployment Shape

TrueFoundry offers all three deployment shapes. The managed SaaS gateway has its control plane hosted in Ireland. In a hybrid setup, you run the gateway plane and TrueFoundry hosts the control plane. In a self-hosted install, the control plane and AI Gateway run as a Helm chart in your own Kubernetes cluster, and air-gapped installs have no outbound network dependencies at runtime.

For the sovereignty questions above, the self-hosted and air-gapped shapes answer them in full. In a self-hosted install, your configuration, secrets, prompts, responses, logs and traces stay in your infrastructure. The software sends outbound-only licensing and analytics metadata to TrueFoundry, such as login emails, hourly usage and cost counts, configuration counts and version information, along with control-plane crash and error logs. User logins pass through TrueFoundry's authentication server unless you connect your identity provider directly. The exact payloads are documented, and prompt and response content is never included.

Running the gateway yourself does not cost you speed. Policy checks run in memory with no external calls in the request path. In TrueFoundry's published benchmark, one gateway pod on a single vCPU added about 3 ms of latency at 250 requests per second and scaled to roughly 350 requests per second before CPU saturation.

Staffbase, an employee experience platform headquartered in Chemnitz, Germany and New York, pools 141 model deployments across nine regions behind one endpoint. It runs separate regional gateways for Europe, the US and Australia, so requests are served from the right geography for residency and latency.

Conclusion

"Hosted in the EU" answers a residency question. Sovereignty asks who operates the system, who holds the keys and what leaves your environment.

If those answers matter to your buyers or your regulator, ask for them in writing and test every vendor against the checklist above, TrueFoundry included. We publish what a self-hosted install sends back, and we would rather you verify it than take our word for it.

Schedule a demo to see a self-hosted TrueFoundry install and its outbound traffic for yourself.