Skip to main content

Documentation Index

Fetch the complete documentation index at: https://www.truefoundry.com/llms.txt

Use this file to discover all available pages before exploring further.

This guide provides instructions for integrating Splunk Observability Cloud with the TrueFoundry AI Gateway to export OpenTelemetry traces.

​
What is Splunk Observability Cloud?

Splunk Observability Cloud is Splunk’s SaaS platform for full-stack observability, including APM and distributed tracing. Sending gateway traces there lets you analyze latency, errors, and dependencies for LLM and tool calls alongside the rest of your stack.

​
Key capabilities

​
Prerequisites

Before integrating Splunk Observability Cloud with TrueFoundry, ensure you have:
  1. TrueFoundry account: Create a TrueFoundry account and follow our Gateway Quick Start Guide
  2. Splunk Observability Cloud org: Access to an org on Splunk Observability Cloud
  3. Access token: An org token (or equivalent) with permission to send trace data, used as the X-SF-Token header (created under Settings → Organization Settings → Access Tokens in the Splunk Observability UI)
The TrueFoundry OTEL Config export sends traces only for the AI Gateway. It does not export OTEL metrics from this screen.

​
Realm and OTLP traces URL

Splunk routes OTLP ingestion by realm (for example us1 or eu0). Your ingest hostname must match the realm for your org. In the URL template below, replace REALM with your Splunk realm (for example us1): 
https://ingest.REALM.observability.splunkcloud.com/v2/trace/otlp
Example for the us1 realm: 
https://ingest.us1.observability.splunkcloud.com/v2/trace/otlp
You can confirm your realm from your Splunk Observability Cloud URL or org documentation.

​
Integration steps

TrueFoundry AI Gateway supports exporting OpenTelemetry traces to Splunk over OTLP/HTTP so you can monitor LLM gateway traffic in Splunk APM.
1

Create an access token in Splunk Observability Cloud

  1. Log in to Splunk Observability Cloud.
  2. Open Settings (gear) → Organization Settings → Access Tokens.
  3. Create a token with Ingest authorization (or a role that includes trace ingest), copy it, and store it securely — you will use it as X-SF-Token in TrueFoundry.
If ingest fails with auth errors, verify the token is active, not expired, and scoped for the same org as your realm.
2

Configure OTEL export in TrueFoundry

  1. In the TrueFoundry dashboard go to Settings → Organisation → AI Gateway → OTEL Config.
  2. Click the edit (pencil) control on the OTEL card.
  3. Enable Otel Traces Exporter Configuration.
  4. Select HTTP Configuration.
  5. Set Endpoint to your Splunk OTLP URL (see Realm and OTLP traces URL).
  6. Set Encoding to Proto.
Settings Organisation AI Gateway OTEL Config in TrueFoundry
3

Configure headers and save

Enable Headers and add:
HeaderValue
X-SF-TokenThe access token from Step 1
Content-Typeapplication/x-protobuf
Click Save to apply the configuration.
TrueFoundry OTEL HTTP exporter for Splunk OTLP endpoint with Proto encoding and headers
4

Verify the integration

  1. Send a few requests through the TrueFoundry AI Gateway.
  2. In TrueFoundry, open Monitor to confirm gateway traffic is flowing.
  3. In Splunk Observability Cloud, go to APM → Trace Analyzer (or your trace explorer) and filter to a recent time range. You should see traces involving tfy-llm-gateway and related downstream spans.
Splunk Observability Cloud Trace Analyzer showing tfy-llm-gateway traces

​
Configuration summary

SettingValue
Traces endpointhttps://ingest.REALM.observability.splunkcloud.com/v2/trace/otlp (replace REALM)
ProtocolHTTP (OTLP)
EncodingProto
HeadersX-SF-Token set to your access token; Content-Type set to application/x-protobuf