What security teams actually need from an AI gateway?

Ask ten people in security what an AI gateway does and you'll get ten answers: a proxy that swaps API keys, a place to log prompts, a firewall for chatbots. None of that is wrong exactly, but none is the full picture, and the gap between "partly right" and "actually governs AI usage" is where breaches happen.

IBM's 2025 Cost of a Data Breach Report found that 20% of breached organizations were compromised through shadow AI, meaning employees using generative AI tools security never approved or knew about, adding roughly $670,000 to the average breach cost. Among those organizations, 97% lacked proper access controls, and a Ponemon Institute survey of 600 organizations found 63% had no AI governance policy at all. [1]

Cost visibility and governance is not a problem that can be solved by buying one more tool. This post walks through it the way a CISO tends to ask: can you see and govern all AI usage in your org, what can you do once you see it, and can it work with the stack you already run.

Question 1: Can you see and govern all the AI usage in your org?

Ask most security leaders to list every AI tool used inside their company, and they can't. AI usage arrives through at least four doors, and each needs a different answer.

Every category of AI client reaches the TrueFoundry AI Gateway through a different path — and gets a different level of governance depending on how cooperative the tool is.

Apps and agents your own developers build. The easiest case, since your teams own the code. Route calls through a scoped, revocable credential instead of a raw API key. TrueFoundry does this with a Virtual Account Token: tied to one application, visible in logs as that application, killable on its own.

Desktop and CLI tools employees already use. Cursor, Claude Code, and Codex CLI typically let you set which endpoint they talk to. That makes enforcement a configuration problem, not an interception problem: push TrueFoundry's gateway endpoint fleet-wide through your MDM (Jamf, Intune).

Consumer web apps. ChatGPT and Claude's web interface are the hard case: no config file, no code to change. TrueFoundry solves this with aitori, an on-device proxy that works like a VPN client, intercepting traffic at the OS level and redirecting it to the gateway via a device certificate. It's open source: the code is on GitHub.

AI embedded inside SaaS tools you already pay for. Salesforce's Einstein, Slack's AI summaries, and similar features are the honest exception. The model call never leaves the vendor's servers, so there's nothing to intercept. No gateway, including TrueFoundry's, solves this today; the fallback is CASB-style controls or the vendor's own admin settings.

Three of four doors are solvable today. The fourth isn't, and any vendor claiming otherwise is worth a second look.

‍