Full-Stack LLM Tracing: Pydantic Logfire and TrueFoundry AI Gateway

The Power of TrueFoundry AI Gateway

TrueFoundry AI Gateway is a unified execution layer for LLM infrastructure. It handles authentication, routing across providers, rate limiting, policy enforcement, MCP tool call management, and — critically for this integration — OpenTelemetry-compliant tracing. Every request through the gateway generates a span carrying standard gen_ai.* attributes (model name, token counts, finish reason) alongside TrueFoundry-specific attributes like tfy.input, tfy.output, and tfy.span_type. These spans are published asynchronously to a NATS message queue after the request completes, meaning the export path never stalls an in-flight request. A dedicated OTEL exporter service reads from that queue and forwards spans to any configured OTLP endpoint over HTTP or gRPC.

Pydantic Logfire: Observability Built for AI

Pydantic Logfire is an observability platform built by the team behind Pydantic — the validation layer embedded in OpenAI's SDK, Anthropic's SDK, and most AI frameworks in production today. Logfire ingests standard OTLP data and applies AI-native rendering on top of it: when it detects gen_ai.* attributes on a span, the LLM Panel activates automatically, surfacing the full conversation history, tool call arguments, per-request token counts, and calculated costs — without any SDK integration on the sending side. Logfire queries are written in PostgreSQL-compatible SQL, so production traces are accessible to humans and to coding agents alike. It is available as a managed cloud service with US and EU regional endpoints.

One Pipeline, Full Visibility: Logfire and TrueFoundry

The integration connects at a single point: TrueFoundry's OTEL Config, which accepts an OTLP HTTP endpoint and an authorization header. Navigate to AI Gateway → Controls → Settings → OTEL Config and click the edit button to open the configuration panel.

‍

TrueFoundry's OTEL Config section — the traces endpoint is pointed at Logfire's EU ingestion URL with the Authorization header set.

Set the endpoint to Logfire's regional ingestion URL, select HTTP with proto encoding, and add the Logfire write token as the Authorization header value. The same write token covers both the traces and metrics exporters.

‍

The Traces Exporter form filled in — endpoint set to https://logfire-eu.pydantic.dev/v1/traces, encoding Proto, and the Logfire write token in the Authorization header.

No code changes are required in the applications sending requests through the gateway. The tracing pipeline operates entirely at the infrastructure layer. A request from any team, using any model, through any provider, generates a span that flows to Logfire carrying the full context of what happened at the gateway.

How Real-Time Tracing Works

When a request arrives at the gateway, the sequence is:

1. Authentication and routing: The gateway validates the caller's JWT, resolves the target model via routing rules (priority, weight, or latency-based), and forwards the request to the selected LLM provider.
2. Response streaming: The provider's response streams back to the client. The gateway records request and response content, token counts, and latency as span attributes including standard gen_ai.* fields.
3. Async span publication: After the response completes, the gateway publishes an OTel span to NATS. This is non-blocking — the client already has its response.
4. OTEL export: The OTEL exporter reads from NATS and delivers the span to Logfire's ingestion endpoint via OTLP HTTP with proto encoding, authenticated via the write token.
5. Logfire rendering: Logfire receives the span and detects gen_ai.* attributes. The LLM Panel activates, displaying the conversation, token usage, cost calculation, and any MCP tool calls that occurred within the same trace.

Once configured, spans from tfy-llm-gateway begin appearing in Logfire's Live view in real time. The tfy.span_type attribute distinguishes ChatCompletion, AgentResponse, and MCPGateway spans — letting teams filter by operation type or query across them in SQL.

‍

Logfire's Live view showing tfy-llm-gateway spans — AgentResponse, ChatCompletion, and MCPGateway operations appear with full timing, status, and nested child spans.

Beyond individual traces, the metrics exporter surfaces aggregate usage data across providers, models, and teams. Logfire's usage overview groups these by scope and span name, giving platform leads a high-level picture of where traffic is going and at what volume.

Logfire's usage overview — metrics from tfy-llm-gateway broken down by instrumentation scope, showing ChatCompletion and MCPGateway traffic across providers.

Get Started with Production-Ready Tracing

Start by creating a write token in Logfire. Navigate to your project, open Project Settings → Write Tokens, and click New write token. Copy the token immediately — Logfire does not show the full value again.

‍

The Logfire Write Tokens page — create a dedicated token for TrueFoundry and store it securely before closing the dialog.

Then go to AI Gateway → Controls → Settings → OTEL Config in TrueFoundry and configure both the traces and metrics exporters with Logfire's regional endpoint and the write token. The full endpoint reference and configuration guide is available in the TrueFoundry documentation. Logfire offers a perpetual free tier, with a self-hosted enterprise option for teams with data residency requirements.

The insight worth taking from this integration is architectural: TrueFoundry and Logfire never needed to coordinate directly. The gateway emits standard OpenTelemetry spans with gen_ai.* attributes; Logfire reads that same standard and activates its LLM-aware views automatically. OpenTelemetry is the contract between them — the gateway governs execution and generates telemetry, Logfire records and visualizes behavior, and the standard connects them without either system depending on the other's internals.

‍

‍