AI Gateway Observability for Agents: Oodle on TrueFoundry

Diseñado para la velocidad: ~ 10 ms de latencia, incluso bajo carga
¡Una forma increíblemente rápida de crear, rastrear e implementar sus modelos!
- Gestiona más de 350 RPS en solo 1 vCPU, sin necesidad de ajustes
- Listo para la producción con soporte empresarial completo
Oodle integration with TrueFoundry AI Gateway: AI observability for every LLM call
Publishing detail
Value
Meta description
The Oodle integration with TrueFoundry AI Gateway sends traces, cost and metrics to Oodle without code changes. See how to set it up and track LLM spend.
URL slug
/blog/oodle-integration-with-truefoundry-ai-gateway
Primary keyword
oodle integration with truefoundry ai gateway
Secondary keywords
AI observability, agent observability, LLM observability, AI agent observability, OpenTelemetry LLM tracing
Link-text keywords
AI gateway observability, LLM cost tracking
SEO title
Oodle integration with TrueFoundry AI Gateway: no SDK needed
Hero image alt text
TrueFoundry and Oodle logos on a co-branded launch banner
Byline
Rohan Ganesh, Partnerships at TrueFoundry
Publish date
November 17, 2026

When an AI agent gives a wrong answer or runs up a surprising bill, the first question is which request caused it. Most teams can't answer that quickly. Every app logs its model calls a little differently, if it logs them at all, and finance wants spend by team in a format nobody planned for. Today we're launching the Oodle integration with TrueFoundry AI Gateway, which answers that question at the one point every model call already passes through.
With export turned on, each LLM call through the gateway appears in Oodle as a trace with its model, latency and cost, next to the rest of your telemetry. That is what AI observability means in practice: a record of what each model and agent call did, how long it took, what it cost and where it failed. A platform admin turns on the export once in the gateway settings, and app teams don't change any code.
What the TrueFoundry AI Gateway does
The TrueFoundry AI Gateway sits between your applications and the models they call. Teams reach 1,600+ models through one OpenAI-compatible API, while the platform team sets routing, fallbacks, guardrails, budgets and rate limits in one place. Our MCP Gateway puts access control and guardrails on the tools that agents call.
Because every request passes through it, the gateway knows things no single app does: which team sent a request, which model served it, whether a guardrail stepped in or a budget blocked it, and what it cost. Our guide to AI gateway observability covers these signals in more depth.
What is Oodle?
Oodle describes itself as "observability built for agents". Alongside metrics, logs, traces, APM, real user monitoring and infrastructure monitoring, it offers AI agent observability: traces, evaluations, prompt management and automated insights. We run alerting for our own gateway and platform on Oodle.
Oodle is built on OpenTelemetry, the open source, vendor-neutral framework for traces, metrics and logs. The gateway can label its spans with OpenTelemetry's semantic conventions for generative AI, the standard names for model calls, so Oodle recognizes them as GenAI traces. Oodle also keeps every trace by default instead of sampling, so the one request a user complains about is still there when you look for it.
Why send AI gateway traces to Oodle
Most AI observability starts inside the application. Each team adds an SDK, and coverage depends on who remembered to. OpenTelemetry LLM tracing at the gateway works differently: the gateway records every model call as a standard trace, so one export setting covers every app and agent that calls a model through TrueFoundry, including the ones nobody instrumented. It also captures events only the gateway sees, such as fallbacks between providers and requests blocked by a budget or rate limit.
Area
TrueFoundry AI Gateway
Oodle
Each request
Records a timed span for each guardrail check, the model call and the provider call
Shows the request as a GenAI trace with model, latency and cost, plus the prompt and response
Cost
Calculates the cost of every request
Prices each call from model rates set in Oodle and breaks down spend for chargeback
Limits and alerts
Enforces budgets and rate limits and records each block
Runs monitors with runbooks and routes alerts to on-call tools
Quality
Applies guardrails and records each result
Scores traffic with LLM-as-judge evaluators and turns failing traces into test datasets
Agent access
Lists the Oodle MCP server and controls who can call which of its tools
Answers agents' queries about logs, metrics, traces and alerts
Setup and data
An admin turns on export once, and TrueFoundry keeps its own copy
Keeps every trace by default instead of sampling
How the Oodle and TrueFoundry integration works

- An app or agent sends its request to the TrueFoundry AI Gateway, exactly as it does today.
- The gateway applies your policies, calls the model and records each step as an OpenTelemetry trace, along with gateway metrics.
- The gateway sends a copy to Oodle asynchronously, off the request path, so a slow or unreachable Oodle can't hold up a model call. TrueFoundry keeps its own copy, so its dashboards don't change.
- In Oodle, teams search and replay requests, track cost and latency, run evaluations and set alerts.
- Optionally, agents query Oodle back through the Oodle MCP server in the MCP Gateway.
Gateway traces cover every model call. For full agent sessions, per-user views and agent graphs, teams also add OpenTelemetry to the app itself. When the app passes its trace context along, the gateway continues the same trace, and the app's spans line up with the gateway's in Oodle.
How to set up the Oodle integration with TrueFoundry AI Gateway
Our Oodle setup guide and Oodle's TrueFoundry guide have the details for each step.
- In Oodle, open the TrueFoundry tile on the Integrations page. It lists the connection details for your Oodle account.
- In TrueFoundry, open the AI Gateway settings, turn on trace export and enter those details.
- If you want gateway metrics in Oodle as well, turn on metrics export the same way.
- Save, send a few requests through the gateway, and check that they appear in Oodle's trace explorer.

In hybrid and self-hosted deployments, the gateway running in your own cluster sends the traces to Oodle itself, and the setup guide covers that path. Gateway traces can include full prompts and responses, so check Oodle's data handling terms before you turn on export.
What teams can do with gateway traces in Oodle
Debug a slow or wrong LLM response
When someone reports a bad answer, you can find the exact request behind it by searching on user, team, error type, guardrail result, request metadata or text in the prompt or response. A gateway trace in Oodle contains:
- the model that served the request, with its latency and cost
- a timed span for each step: the incoming request, each guardrail check, the model call and the provider call
- the result of each guardrail check and any error
- the user, team and other metadata attached to the request
- the prompt and the response
We verified the first two on gateway traffic in our test environment. If a fallback kicked in, Oodle shows each attempt as its own span under the same request, and you can open the trace in Oodle's playground to replay the conversation and test a fix.
Track LLM cost by team, user and model
We compared Oodle's cost figures with the gateway's own in our test environment, and the per-request figures matched. On one test request, Oodle showed $0.030 and the gateway $0.029958, the same amount rounded to a tenth of a cent.
Oodle prices each call from the model rates you set in Oodle, so keep those in line with your provider pricing. From there you can split spend by model, team, user or any metadata label for chargeback, and get an alert when spending climbs. For the finance side, see our guide to LLM cost tracking.
Monitor LLM latency, errors and rate limits
In our tests, these gateway metrics arrived in Oodle next to the traces:
- requests, HTTP status codes and cost
- latency, time to first token and inter-token latency
- budget and rate-limit checks, and the requests they blocked
- MCP request counts and latency
Oodle monitors can alert on any of these and send the alert to Slack, PagerDuty, Opsgenie, Rootly, Google Chat or Microsoft Teams with a runbook attached.
Evaluate LLM quality on production traffic
Oodle can flag behavior as traces arrive, such as refusals or a user asking the same thing again without getting an answer, and you can describe a custom signal in plain English. It also runs LLM-as-judge evaluators for issues like hallucination and toxicity.
Teams can add failing traces to a dataset and test a new prompt or model against those real cases before it ships. We haven't tested these features on gateway traffic yet, so Oodle's docs are the reference for how they behave.
Let AI agents query Oodle through the MCP Gateway

The Oodle MCP server is listed among the official remote MCP servers in our MCP Gateway. Agents behind the gateway can use it to query logs, metrics, traces and alerts in Oodle and to work with Grafana dashboards. An on-call agent, for example, can list active alerts and pull the traces behind them.
With OAuth sign-in, each user connects their own Oodle account, so an agent sees only what that person can see in Oodle (an Oodle API key works too). The gateway decides who can use which Oodle tools, applies guardrails to those calls and traces each one with the caller's identity.

FAQ
What is the Oodle integration with TrueFoundry AI Gateway?
It connects the two products in two ways, and each works on its own. The gateway exports OpenTelemetry traces and metrics for every request to Oodle, where teams debug calls and track cost. Separately, Oodle's MCP server is available in TrueFoundry's MCP Gateway, so agents can query Oodle under the gateway's access controls.
What's the difference between LLM observability and agent observability?
LLM observability tracks individual model calls: the prompt, the response, latency, errors and cost. That is enough to debug a single slow or wrong answer. Agent observability follows a whole agent run across many model and tool calls to explain why the agent did what it did. Gateway export covers every model call, and instrumenting the agent itself joins those calls into full agent traces in Oodle.
What latency does the TrueFoundry AI Gateway add?
Very little. The gateway adds under 3 ms of internal latency, and in our published benchmark, run on a machine with 1 vCPU and 1 GB of memory, it handled 350+ RPS. Telemetry export to Oodle runs off the request path, so an Oodle outage can't block a model call.
Does TrueFoundry support MCP and AI agents?
Yes. The MCP Gateway sits in front of the MCP servers your agents use, including Oodle's. It controls who can call which tool and applies guardrails to those calls, and it traces each call with the caller's identity. Gateway metrics in Oodle include MCP request counts and latency, so tool traffic shows up next to model traffic.
Does TrueFoundry work with my existing observability stack?
Yes. The gateway exports standard OpenTelemetry traces and metrics, so it works with Oodle and with other OpenTelemetry-compatible backends you may already run. Our export docs explain how to connect them.
Next steps and related reading
Turning on the Oodle integration with TrueFoundry AI Gateway is a one-time change in the gateway settings that needs no application code. From then on, every model call through the gateway reaches Oodle as a trace with its cost attached, where engineers can debug it and finance can report on it.
Book a demo to see the gateway and Oodle working together on your own traffic, or explore the TrueFoundry AI Gateway. To set it up now, follow our Oodle setup guide or Oodle's TrueFoundry guide, and start at oodle.ai if you don't have an Oodle account yet.
- What is LLM observability?: the signals to track for model calls
- OpenTelemetry for LLMs: how we instrument a multi-provider AI gateway: span design, cost attribution and async export
- Top 5 Helicone alternatives: how observability-focused LLM gateways compare
Editorial notes
Before publishing
- Yellow highlight marks stand-ins filled in for Rohan to confirm or replace: the byline and role, the publish date, the "1,600+ models" and "under 3 ms" figures (the /ai-gateway page shows both 1,600+ and 250+ models, and the blog skill lists 1,000+ and 3 to 4 ms), and the sentence about running our own alerting on Oodle.
- Get Oodle's sign-off on the draft, the byline and the co-publishing plan.
- Confirm how SaaS customers turn on the OpenTelemetry GenAI attributes that Oodle's AI views use.
- Oodle currently over-counts tokens on gateway traces (reported to Oodle on Nov 2). Use no token-total screenshots or token-accuracy claims until it's fixed.
- Use a clean demo request for any screenshots, never devtest data.
- Rename the MCP catalog entry from oodle-ai to oodle (Oodle's request on Nov 6).
- Proof point now used in the "What is Oodle?" section, needs sign-off: TrueFoundry runs its own gateway and platform alerting on Oodle.
- Screenshots: the Oodle instance ID, the API key values and Oodle's OAuth client ID are blurred. Oodle's Connect TrueFoundry page still shows the oodle-ai name, which Oodle asked us to change, so get their OK on it.
- Alt text, telemetry diagram: TrueFoundry AI Gateway sending OpenTelemetry traces and metrics to Oodle asynchronously
- Alt text, export settings: TrueFoundry AI Gateway OpenTelemetry trace and metrics export settings pointed at Oodle
- Alt text, MCP diagram: AI agents querying the Oodle MCP server through the TrueFoundry MCP Gateway with per-user OAuth sign-in
- Alt text, Oodle screenshot: Oodle's Connect TrueFoundry page with steps to add the Oodle MCP server to the TrueFoundry MCP Gateway
- Full-size image files for the CMS are in the Oodle blog images folder in Drive.
Keyword research
These estimates come from Google Trends (US, last 12 months, each term compared against "llm gateway"). They were converted to monthly searches using two Semrush volumes from the BOFU keyword sheet ("llm gateway" 720 and "mcp gateway" 880, treated as 2025 averages and brought forward with each term's own Trends history), and checked against Google Autocomplete and live search results. Semrush itself was not used.
The order between terms is reliable. The larger terms are probably within ±50%, and terms under about 300 searches a month are too thin for Trends to measure.
Term
Searches a month now (US)
12-month average
Last 6 months
Ranking difficulty
Use in post
agent observability
~2,400 (1,900 to 3,000)
~3,200
Rising, 2.1×
Very hard
Oodle section heading
ai observability
~2,300 (1,800 to 2,900)
~5,300
Steady
Very hard
H1 and meta description
llm evaluation
~2,600 (2,100 to 3,300)
~9,900
Easing
Not checked
Mention only
llm observability
~800 (650 to 1,050)
~5,900
Falling after an early-2026 peak
Very hard
Intro and FAQ
ai agent observability
~250 (200 to 350)
~960
Rising, 2.7×
Very hard
Body text
llm monitoring
~200 (150 to 250)
~750
Steady
Hard
Skip
ai observability tools
~220
~1,100
Falling
Very hard (listicles)
Skip
llm observability tools
Under ~100
~310
Falling
Very hard (listicles)
Skip
opentelemetry llm
Under ~300
~620
Steady
Medium
How it works
llm tracing
Under ~300
~350
Steady
Medium
Body text
llm cost tracking
Under ~300
~510
Falling
Medium
Link text only
ai gateway observability
Under ~200
~220
Rising from a tiny base
Medium
Link text only
agentic observability
Under ~300
~520
Steady
Not checked
Skip
truefoundry ai gateway
Under ~100
n/a
n/a
Not checked
Body text
oodle ai
Under ~100
n/a
n/a
Easy
"Oodle (oodle.ai)" on first mention
oodle integration with truefoundry ai gateway
Under ~100
n/a
n/a
Easy, nothing ranks yet
Primary keyword
Where TrueFoundry already ranks:
- AI gateway observability: the existing "Observability in AI gateways" post is about #3.
- LLM cost tracking: the "LLM cost tracking solution" post is around #20.
- "oodle ai observability": TrueFoundry's Oodle docs page already shows up.
- AI observability and LLM observability: only translated copies of TrueFoundry pages appear, around #15. IBM, Dynatrace, Splunk and Elastic hold the top spots, so this post uses those terms for relevance rather than to rank.
- Google mixes up "oodle" with the unrelated company Oodles, which is why the post writes "Oodle (oodle.ai)" on first mention.
TrueFoundry AI Gateway ofrece una latencia de entre 3 y 4 ms, gestiona más de 350 RPS en una vCPU, se escala horizontalmente con facilidad y está listo para la producción, mientras que LitellM presenta una latencia alta, tiene dificultades para superar un RPS moderado, carece de escalado integrado y es ideal para cargas de trabajo ligeras o de prototipos.














.webp)
.webp)


.webp)
.webp)

.webp)
.webp)
.png)

.png)
.png)






