agentgateway (formerly Gloo AI Gateway): an honest review of Solo.io's AI gateway

Search for Gloo AI Gateway today and solo.io sends you somewhere else: the old product page at solo.io/products/gloo-ai-gateway redirects to agentgateway. Behind the redirect, Solo moved its AI gateway features off Envoy and onto a new proxy built for MCP and agent traffic.

That much is in Solo's announcements. We wanted to know what the gateway actually does, so we ran it ourselves and checked the behaviours that matter in production, with version numbers wherever a result depends on them. If you're comparing Solo against the wider field, our roundup of Solo.io competitors and alternatives covers that ground. This post stays on the gateway itself.

agentgateway (formerly Gloo AI Gateway) review at a glance

What happened to Gloo AI Gateway

Solo's AI gateway features used to run on Gloo Gateway, Solo's Envoy-based API gateway. Solo donated the open-source Gloo Gateway to the CNCF, where it was renamed kgateway and accepted as a Sandbox project in March 2025. That same month Solo created a new proxy from scratch in Rust: the agentgateway repository dates from 18 March 2025.

The handover took about eight months, across two kgateway releases, one Solo announcement and a move to a foundation:

On the commercial side, Gloo Gateway was renamed Solo Enterprise for kgateway, and Solo Enterprise for agentgateway, once included in Gloo Gateway, became a separate product with its own Helm charts, controllers and APIs.

Existing configuration doesn't carry over unchanged. kgateway v2.2 required migrating agentgateway settings to the new resources before upgrading, and the Solo Enterprise 2.1 rename removed "Gloo" from CRDs and Helm charts.

What is agentgateway?

agentgateway is an open source AI gateway, self-hosted, that sits between your applications and the model providers, tools and agents they call. Applications talk to it with the standard OpenAI request format, so adopting it is mostly a base-URL change. It governs three kinds of traffic: LLM calls, Model Context Protocol (MCP) traffic between agents and tools, and agent-to-agent (A2A) calls. (For why A2A traffic needs its own controls, see our post on governing multi-agent A2A traffic at the gateway.) It also proxies ordinary HTTP and gRPC. (What is an LLM gateway? covers the general pattern.) For the wider category, see our guide to agent gateways.

Solo's docs explain the rewrite: MCP sessions are long-lived and stateful, and traditional gateways were not designed for that kind of session awareness, so Solo built a new proxy instead of extending Envoy.

Version History

Minor releases ship roughly monthly, which means regular upgrade work. v1.6.0 alone had five breaking changes, and two of them are in the migration checklist below.

How we tested

We ran the open-source standalone binary, v1.5.0, against local mock providers, plus a few real OpenAI calls where the provider itself was under test. When v1.6.0 shipped, we re-ran the four findings it was most likely to change on both versions. We haven't run the commercial edition, which needs a sales-issued licence key, so what we say about it comes from Solo's documentation.

Architecture

There are two ways to run it.

Standalone, it's one process reading one YAML file. It needs no Kubernetes, Postgres or Redis to route LLM traffic, and it includes a web UI and an embedded SQLite request log. A minimal LLM configuration is under ten lines, with no account to create.

On Kubernetes, a controller watches Gateway API resources and pushes configuration to separate proxy deployments over xDS. agentgateway has its own GatewayClass and CRDs (AgentgatewayPolicy, AgentgatewayBackend, AgentgatewayParameters). Solo's optional Helm values for ClickHouse, Redis and OpenTelemetry are outside its support policy.

What we checked hands-on on v1.5.0:

• Control-plane loss. We deleted the controller in a local Kubernetes cluster while traffic was flowing. The proxies kept serving from their existing configuration with zero failed requests across a 66-second outage, and killing the active controller twice moved leadership to the standby in 16 and 17 seconds.
• No phone-home. On a network with no internet route, the gateway served traffic end to end and made no outbound connections beyond the configured providers. We saw no licence check, telemetry callback or usage reporting.
• Live reload. Routing rules, policies, backends and even listener ports reload in about two seconds, without a restart. An invalid configuration is rejected with an error naming the field, and the last good configuration keeps serving traffic.
• The exception to live reload. A few settings, such as the request-log database location, only apply at startup. Change one while the gateway runs and it logs that it loaded the new configuration, but nothing changes, and its config-sync metric still reports in sync.

Key capabilities, as documented and tested

Providers and formats

The v1.6.0 schema has seven built-in provider types (OpenAI, Gemini, Vertex AI, Anthropic, Amazon Bedrock, Azure and GitHub Copilot) and 14 presets, including Groq, Mistral, DeepSeek, Together AI, Ollama and OpenRouter. A custom provider type reaches any other host that speaks one of the nine supported API formats, so in practice the reach is any OpenAI-compatible endpoint.

Passthrough fidelity was excellent in our tests. We sent tool calls, strict JSON-schema structured outputs, system prompts, image inputs, logprobs and seeded requests both directly to OpenAI and through the gateway, and the results matched. A separate check for "silent breaks", where a request returns HTTP 200 with empty or wrong content, found none across the capabilities we tested.

Image generation and audio are the gap. The LLM-aware path doesn't recognise image generation, transcription or speech requests. You can still proxy that traffic as plain HTTP, and we confirmed it arrives intact with API-key checks applied, but with no token accounting, cost tracking or model-aware policy.

Routing and resilience

agentgateway supports weighted routing, priority-based failover, conditional routing, retries, timeouts, outlier eviction, mirroring and session affinity. Failover works across providers with different API formats: we failed a chat-completions backend over to an Anthropic-format backend, and the client kept receiving responses in the shape it asked for.

Three behaviours need deliberate configuration.

Retries make one more upstream call than the schema describes (v1.5.0 and v1.6.0). The schema describes attempts as the "total number of attempts, including the original request." In our tests, attempts: 1, 2 and 3 produced 2, 3 and 4 backend calls. The gateway's own retry.attempt log field counts the same way, so the behaviour is consistent and only the documentation disagrees. Retries also have no backoff unless you configure one. Plan for one extra call per failure, and set a backoff.

Failover needed health checks in v1.5.0, and v1.6.0 fixes this. In v1.5.0, failover didn't move traffic off a failing primary unless an eviction policy was configured. On v1.6.0 with no health policy, the primary was evicted after one failure and the next five requests went to the backup. On both versions, the request that hit the failure still reached the client as an error and wasn't retried on the backup.

Stalled streams need an explicit timeout. In v1.5.0, neither requestTimeout nor backendRequestTimeout ended a stream that sent its first chunk and then went silent. v1.6.0 adds responseIdleTimeout, and set to 5 seconds, it cut our stalled stream at 5.04 seconds. The client has already received a 200 and the first chunk by then, so it sees a truncated stream, not an error code. With no timeout configured, the stream still hangs on v1.6.0 for as long as the client waits. This is the route policy we tested:

routes:
- matches:
  - path:
      pathPrefix: /
  policies:
    timeout:
      responseIdleTimeout: 5s
  backends:
  - host: "127.0.0.1:8091"

Identity, access and MCP authorization

Callers authenticate with API keys or with JWTs validated against your identity provider. Keys carry metadata such as user and team, and that metadata reaches the request log. Per-key model allow-lists work in the open-source build, and revoking a key took effect in under two seconds in our tests.

MCP authorization is the strongest part of the product, and ahead of most of the category. CEL (Common Expression Language) policies decide which individual tools a caller may invoke, and the gateway also filters tools/list per caller, so an agent only sees the tools it's allowed to use. We confirmed both hands-on. If you're new to how MCP identity works, MCP authentication explained is a good primer. agentgateway can also federate several MCP servers behind one endpoint and expose OpenAPI services as MCP tools.

Enforcement worked the same way for LLM, MCP and A2A traffic in our tests. One caveat: caller identity isn't written to the access log for MCP and A2A traffic unless you configure it.

Spend controls and rate limits

In the open-source standalone build (v1.5.0), API keys can carry USD or token budgets, and we confirmed that one key hitting its limit is blocked while other keys carry on. Enforcement is retrospective. The gateway checks the budget before admitting a request but only learns the cost after the response, so the request that crosses the limit still completes. Treat budgets as a near-real-time brake rather than a hard cap. (We cover the general trade-offs in AI gateway budgets and quotas.)

Cost depends on a price catalog. In v1.5.0, no cost is calculated until you load one, and a budget on a model the catalog doesn't price never charges and never blocks. v1.6.0 includes a built-in catalog that priced gpt-4o-mini with nothing configured, but a self-hosted or in-house model still recorded no cost. We haven't re-tested budget enforcement on v1.6.0.

Rate limits in the open-source build are local to each gateway instance, and v1.6.0 adds per-key local limits, which we haven't tested. Global rate limiting, the budget resource and the Cost Management UI are enterprise features. We tested budgets on one instance only, so test how they behave across several standalone instances before relying on them.

Caching

We found no response caching. We searched the configuration schema, the open-source docs and Solo's enterprise docs for exact-match and semantic caching and found neither. Teams with repetitive prompts will need to cache somewhere else.

Guardrails and data protection

Built-in detectors replace sensitive data such as email addresses, card numbers and national ID numbers with labelled placeholders. With a recording mock where the provider would be, the masked values were already gone before the request left the gateway. Requests can also be rejected with a custom message. External guards include webhooks, OpenAI Moderation, AWS Bedrock Guardrails and Google Model Armor, which we didn't test.

Prompt and response content isn't recorded by default. When you do turn recording on, CEL rules can limit it to particular teams, costs or guardrail outcomes.

Observability

Prometheus metrics cover requests, latency, tokens and cost. A request log in SQLite or Postgres records user, team, model, tokens and cost per request, so per-user reporting is a SQL query. Traces and logs export over OpenTelemetry, and Solo documents LangSmith and Phoenix integrations.

Per-team cost isn't in the metrics, though, which matters if AI gateway observability is how you plan to report spend. Prometheus metrics carry no user, team or key label, so a cost-by-team dashboard has to read the request log. That's a reasonable choice for cardinality reasons, but it means building a second data path.

Who runs agentgateway

Solo.io, based in Cambridge, Massachusetts, builds agentgateway and sells the enterprise edition. Its last announced funding round was a $135M Series C at a $1B valuation in October 2021.

The project itself is hosted by the Agentic AI Foundation under the Linux Foundation, not by Solo. The code is Apache 2.0, so if Solo's priorities change, the code and the governance structure carry on.

Day to day, most of the code still comes from Solo. Of the repository's 2,507 non-merge commits on 25 August 2026, solo.io email addresses made up 63.4% (a floor, since staff who commit from personal addresses aren't counted) and one Solo engineer authored 38.9%. In the previous 90 days, 118 people committed and Solo's share was 58.9%, with Microsoft contributing Windows support and AWS Bedrock examples. That concentration is normal for a project about 19 months old, and outside contributions are growing. A two-year bet should still weigh how much of the project rests on Solo, and on one engineer in particular.

Migrating from Gloo AI Gateway: a checklist

If you ran AI policies on Gloo Gateway or the Envoy path of kgateway, this is the order we'd work in. Every item comes from the release notes and behaviours described above.

1. Inventory the AI policy you have on the Envoy path's TrafficPolicy. kgateway v2.2 removed it, so nothing there survives the upgrade.
2. Move that configuration to the agentgateway GatewayClass and its CRDs (AgentgatewayPolicy, AgentgatewayBackend, AgentgatewayParameters) before you upgrade to v2.2, not after.
3. On the commercial track, update manifests, Helm values and automation that reference "Gloo" names. Solo Enterprise 2.1 removed them from CRDs and Helm charts.
4. On agentgateway v1.6.0 or later, add the path to every provider base URL (https://api.openai.com/v1, not https://api.openai.com).
5. If you convert Anthropic Messages requests for OpenAI-style providers, check whether you depend on extended-thinking history, which the Responses conversion drops.
6. Set a retry backoff, and budget for one more upstream call per failure than attempts suggests.
7. Set responseIdleTimeout on streaming routes so a silent provider can't hold connections open.
8. Confirm every model you route, including self-hosted ones, is priced in the catalog. Unpriced models record no cost and never trip a budget.
9. Decide which side of the edition boundary you need. Global rate limits, the budget CRD and the Cost Management UI are enterprise features.

Open source vs Solo Enterprise

Solo's feature matrix marks the enterprise additions below.

Parts of Solo's feature matrix undersell the free product. It lists spend budgets and virtual keys under the enterprise Cost Management UI, yet the open-source standalone binary enforced per-key budgets in our testing. Part of the reason is that the open-source Kubernetes CRDs have no budget or virtual-key fields, while the standalone binary does. We haven't had Solo confirm the rest, so test the edition boundary before you price around it.

Pricing and compliance

Buying is sales-led, with no public price list, and licence keys come from an account representative. The only outside figure we found is Vendr's median Solo.io contract of $68,000 a year (range $28,954 to $530,656), which covers Solo.io as a whole, not agentgateway specifically.

Solo's trust portal shows SOC 2 Type II. When we checked, it listed neither ISO 27001 nor a HIPAA business associate agreement. A missing listing doesn't prove the paperwork is unavailable, so regulated buyers should ask Solo directly.

When agentgateway is a good fit

agentgateway fits a platform team that wants to own its gateway and runs one or a few instances, agent workloads where MCP tool authorization matters most, and environments with no internet route. It also suits teams that put real weight on an Apache 2.0 licence and foundation governance.

It's a poor fit if you need budgets and rate limits shared across many instances without buying the enterprise edition, or response caching in the gateway. The same goes for teams that want a hosted service, can't absorb monthly upgrades with breaking changes, or need cost tracking on image generation and audio.

How TrueFoundry approaches this

The TrueFoundry AI Gateway is the same kind of control point, with the surrounding platform included. It runs as SaaS or self-hosted in your VPC, on-prem or air-gapped, and exposes 1,000+ LLMs through one OpenAI-compatible API. Around the gateway it adds budgets scoped by tenant, team, model or provider account with a warn-only mode, a console with RBAC, SSO over OIDC or SAML 2.0 and SCIM provisioning, and MCP governance with per-tool controls and human approval before destructive tool calls.

One caveat: TrueFoundry's content moderation, PII and prompt-injection guardrails work only when TrueFoundry hosts the gateway, while agentgateway's built-in PII masking runs on your own infrastructure.

agentgateway vs TrueFoundry AI Gateway

The TrueFoundry column restates TrueFoundry's own published documentation. The agentgateway column reflects our testing of the open-source build and Solo's documentation for the enterprise edition, as of 8 October 2026.

Which one fits

Choose Solo Enterprise for agentgateway if you're committed to agentgateway's data plane but need fleet-wide rate limits, a cost UI, FIPS builds and a support contract.

Choose TrueFoundry if you'd rather not assemble budgets, access control and tool approvals around a gateway yourself, or if you want the option of a managed SaaS deployment alongside VPC and air-gapped installs.

Related reading

• Envoy AI Gateway (now Agent Router): an honest review, our review of the other foundation-hosted open-source AI gateway
• Solo.io competitors and alternatives, the wider market around Solo's products
• What is an Agent Gateway? A Complete Guide (2026), our guide to the wider agent gateway category
• Kong AI vs Solo.io, comparing Solo with Kong's AI gateway
• LiteLLM alternatives, for teams weighing open-source proxies
• Best MCP gateways, comparing the MCP layer across vendors

Conclusion

Gloo AI Gateway as a name is gone. Solo's AI gateway is now agentgateway, a proxy written for LLM, MCP and agent traffic from the start, faithful to provider APIs and more generous in its free edition than Solo's own matrix admits. Before you build on it, test where the free edition stops, set a retry backoff and an idle timeout, and check that every model you use is priced in the catalog.

If you're weighing agentgateway against a gateway with team budgets, access control and MCP approvals already built in, book a demo of the TrueFoundry AI Gateway and bring your MCP and budget requirements. You can also read how the gateway works first.