gen_ai.*), alongside related standard OTel attributes such as error.type and server.*. This lets GenAI-aware observability platforms — and any collector that understands the GenAI conventions — consume gateway traces without a TrueFoundry-specific attribute translator, and lets you join gateway spans with spans your own SDK instrumentation already produces.
This is a dual-write: gen_ai.* attributes are added in addition to the tfy.* attributes. No tfy.* attribute is removed or renamed, so existing dashboards, queries, and integrations continue to work unchanged. Gateway-specific concepts that have no GenAI equivalent — cost in USD, tenant and subject, semantic cache, guardrails, load balancing, budgets, and rate limits — remain available only under tfy.*.
Enabling GenAI Attributes
This feature is behind a feature flag and is off by default. To turn it on, set the following environment variable on yourtfy-llm-gateway deployment and restart it:
The OpenTelemetry GenAI semantic conventions are still in development upstream, so attribute names may change in future releases.
tfy.* attributes remain the stable source of truth.Attribute Mapping
Eachgen_ai.* attribute is written only when the corresponding source data is present on the span. The sampling parameters (gen_ai.request.temperature through gen_ai.request.presence_penalty) are extracted from the request body and appear on model spans only.
Content and Privacy
gen_ai.input.messages and gen_ai.output.messages are derived from tfy.input and tfy.output after redaction has been applied, so redaction rules carry over. If prompt logging is disabled, the source attributes are removed and these GenAI attributes are absent entirely.
Payload-size limits are applied afterwards and evaluated independently for each attribute. If a message payload exceeds the limit, the attribute is present but set to the sentinel value { "__skipped_large_payload": true } rather than being dropped. Because the GenAI message format adds wrapping JSON, gen_ai.input.messages can be skipped for size while tfy.input is still retained.
Current Limitations
The initial release covers inference-critical fields on model and root spans, though some attributes are narrower than that — see the notes in the mapping table. Agent attributes (gen_ai.agent.*), tool call attributes (gen_ai.tool.*), retrieval attributes, and MCP attributes (mcp.*) are not yet dual-written — that data remains available under tfy.agent.*, tfy.model.tool_calls, and tfy.mcp_server.*. Prometheus metric labels are unaffected by this flag and continue to use TrueFoundry names.