agentgateway (formerly Gloo AI Gateway): an honest review of Solo.io's AI gateway

Search for Gloo AI Gateway today and solo.io sends you somewhere else: the old product page at solo.io/products/gloo-ai-gateway redirects to agentgateway. Behind the redirect, Solo moved its AI gateway features off Envoy and onto a new proxy built for MCP and agent traffic.

That much is in Solo's announcements. We wanted to know what the gateway actually does, so we ran it ourselves and checked the behaviours that matter in production, with version numbers wherever a result depends on them. If you're comparing Solo against the wider field, our roundup of Solo.io competitors and alternatives covers that ground. This post stays on the gateway itself.

agentgateway (formerly Gloo AI Gateway) review at a glance

What happened to Gloo AI Gateway

Solo's AI gateway features used to run on Gloo Gateway, Solo's Envoy-based API gateway. Solo donated the open-source Gloo Gateway to the CNCF, where it was renamed kgateway and accepted as a Sandbox project in March 2025. That same month Solo created a new proxy from scratch in Rust: the agentgateway repository dates from 18 March 2025.

The handover took about eight months, across two kgateway releases, one Solo announcement and a move to a foundation:

On the commercial side, Gloo Gateway was renamed Solo Enterprise for kgateway, and Solo Enterprise for agentgateway, once included in Gloo Gateway, became a separate product with its own Helm charts, controllers and APIs.

Existing configuration doesn't carry over unchanged. kgateway v2.2 required migrating agentgateway settings to the new resources before upgrading, and the Solo Enterprise 2.1 rename removed "Gloo" from CRDs and Helm charts.

What is agentgateway?

agentgateway is an open source AI gateway, self-hosted, that sits between your applications and the model providers, tools and agents they call. Applications talk to it with the standard OpenAI request format, so adopting it is mostly a base-URL change. It governs three kinds of traffic: LLM calls, Model Context Protocol (MCP) traffic between agents and tools, and agent-to-agent (A2A) calls. (For why A2A traffic needs its own controls, see our post on governing multi-agent A2A traffic at the gateway.) It also proxies ordinary HTTP and gRPC. (What is an LLM gateway? covers the general pattern.) For the wider category, see our guide to agent gateways.

Solo's docs explain the rewrite: MCP sessions are long-lived and stateful, and traditional gateways were not designed for that kind of session awareness, so Solo built a new proxy instead of extending Envoy.

Minor releases ship roughly monthly, which means regular upgrade work. v1.6.0 alone had five breaking changes, and two of them are in the migration checklist below.

How we tested

We ran the open-source standalone binary, v1.5.0, against local mock providers, plus a few real OpenAI calls where the provider itself was under test. When v1.6.0 shipped, we re-ran the four findings it was most likely to change on both versions. We haven't run the commercial edition, which needs a sales-issued licence key, so what we say about it comes from Solo's documentation.

Architecture

There are two ways to run it.

‍

Standalone, it's one process reading one YAML file. It needs no Kubernetes, Postgres or Redis to route LLM traffic, and it includes a web UI and an embedded SQLite request log. A minimal LLM configuration is under ten lines, with no account to create.

‍

On Kubernetes, a controller watches Gateway API resources and pushes configuration to separate proxy deployments over xDS. agentgateway has its own GatewayClass and CRDs (AgentgatewayPolicy, AgentgatewayBackend, AgentgatewayParameters). Solo's optional Helm values for ClickHouse, Redis and OpenTelemetry are outside its support policy.

‍

‍

Figure 2: Standalone and Kubernetes deployments share one data plane and the same upstreams.

‍

What we checked hands-on on v1.5.0:

‍

• Control-plane loss. We deleted the controller in a local Kubernetes cluster while traffic was flowing. The proxies kept serving from their existing configuration with zero failed requests across a 66-second outage, and killing the active controller twice moved leadership to the standby in 16 and 17 seconds.
• No phone-home. On a network with no internet route, the gateway served traffic end to end and made no outbound connections beyond the configured providers. We saw no licence check, telemetry callback or usage reporting.
• Live reload. Routing rules, policies, backends and even listener ports reload in about two seconds, without a restart. An invalid configuration is rejected with an error naming the field, and the last good configuration keeps serving traffic.
• The exception to live reload. A few settings, such as the request-log database location, only apply at startup. Change one while the gateway runs and it logs that it loaded the new configuration, but nothing changes, and its config-sync metric still reports in sync.

Key capabilities, as documented and tested

Providers and formats

The v1.6.0 schema has seven built-in provider types (OpenAI, Gemini, Vertex AI, Anthropic, Amazon Bedrock, Azure and GitHub Copilot) and 14 presets, including Groq, Mistral, DeepSeek, Together AI, Ollama and OpenRouter. A custom provider type reaches any other host that speaks one of the nine supported API formats, so in practice the reach is any OpenAI-compatible endpoint.

‍

Passthrough fidelity was excellent in our tests. We sent tool calls, strict JSON-schema structured outputs, system prompts, image inputs, logprobs and seeded requests both directly to OpenAI and through the gateway, and the results matched. A separate check for "silent breaks", where a request returns HTTP 200 with empty or wrong content, found none across the capabilities we tested.

‍

Image generation and audio are the gap. The LLM-aware path doesn't recognise image generation, transcription or speech requests. You can still proxy that traffic as plain HTTP, and we confirmed it arrives intact with API-key checks applied, but with no token accounting, cost tracking or model-aware policy.

Routing and resilience

agentgateway supports weighted routing, priority-based failover, conditional routing, retries, timeouts, outlier eviction, mirroring and session affinity. Failover works across providers with different API formats: we failed a chat-completions backend over to an Anthropic-format backend, and the client kept receiving responses in the shape it asked for.

‍

Three behaviours need deliberate configuration.

‍

Retries make one more upstream call than the schema describes (v1.5.0 and v1.6.0). The schema describes attempts as the "total number of attempts, including the original request." In our tests, attempts: 1, 2 and 3 produced 2, 3 and 4 backend calls. The gateway's own retry.attempt log field counts the same way, so the behaviour is consistent and only the documentation disagrees. Retries also have no backoff unless you configure one. Plan for one extra call per failure, and set a backoff.

‍

Failover needed health checks in v1.5.0, and v1.6.0 fixes this. In v1.5.0, failover didn't move traffic off a failing primary unless an eviction policy was configured. On v1.6.0 with no health policy, the primary was evicted after one failure and the next five requests went to the backup. On both versions, the request that hit the failure still reached the client as an error and wasn't retried on the backup.

‍

Stalled streams need an explicit timeout. In v1.5.0, neither requestTimeout nor backendRequestTimeout ended a stream that sent its first chunk and then went silent. v1.6.0 adds responseIdleTimeout, and set to 5 seconds, it cut our stalled stream at 5.04 seconds. The client has already received a 200 and the first chunk by then, so it sees a truncated stream, not an error code. With no timeout configured, the stream still hangs on v1.6.0 for as long as the client waits. This is the route policy we tested:

‍