Skip to main content

Documentation Index

Fetch the complete documentation index at: https://www.truefoundry.com/llms.txt

Use this file to discover all available pages before exploring further.

This guide provides instructions for integrating PromptLayer with the TrueFoundry AI Gateway to export OpenTelemetry traces.

​
What is PromptLayer?

PromptLayer is an LLM observability and prompt management platform that accepts OpenTelemetry traces over OTLP/HTTP. It ingests GenAI-standard OTLP spans and converts them into structured trace views, request logs, and analytics dashboards for monitoring and debugging LLM workflows.

​
Key Features of PromptLayer

  • Trace Observability: Ingests OTLP spans and renders them as LLM call traces with inputs, outputs, token usage, and model metadata — visible under the Traces section of the dashboard.
  • Prompt Management: Version-controlled prompt registry that links deployed prompts to observed traces for side-by-side comparison and evaluation.
  • Analytics & Evaluation: Aggregated dashboards for latency, cost, and quality across LLM providers and prompt versions.

​
Prerequisites

Before integrating PromptLayer with TrueFoundry, ensure you have:
  1. TrueFoundry Account: Create a TrueFoundry account and follow the instructions in our Gateway Quick Start Guide.
  2. PromptLayer Account: Sign up at promptlayer.com and have access to your workspace to retrieve an API key.

​
Integration Steps

1

Get Your PromptLayer API Key

  1. Log into your PromptLayer workspace.
  2. Click Settings in the bottom-left sidebar.
  3. Select PromptLayer API Keys under the Workspace section.
  4. Click Generate New API Key and copy the key.
PromptLayer Settings page showing the PromptLayer API Keys section with existing keys listed
Create separate keys for different environments (e.g. prod and dev) so you can rotate them independently without affecting all integrations.
2

Configure OTEL Export in TrueFoundry

  1. Go to AI Gateway → Settings in the TrueFoundry dashboard.
  2. Scroll down to the OTEL Config section and click the edit (✏️) button.
TrueFoundry AI Gateway Settings page showing the OTEL Config section with Traces Endpoint and Headers highlighted
  1. Enable the Otel Traces Exporter Configuration toggle and fill in:
FieldValue
ToggleEnabled
ProtocolHTTP Configuration
Endpointhttps://api.promptlayer.com/v1/traces
EncodingProto
Header KeyX-API-KEY
Header Value<your-promptlayer-api-key>
TrueFoundry Configure Settings for OTEL Config modal showing the Traces Exporter filled in with the PromptLayer endpoint and X-API-KEY header
  1. Click Save to apply the configuration.
PromptLayer only accepts OTLP traces via its /v1/traces endpoint. It does not expose an OTLP metrics ingestion endpoint, so the Metrics Exporter should be left disabled. Make sure the header key is entered as X-API-KEY in all caps — PromptLayer’s API enforces this exact casing.
3

Verify the Integration

  1. Make a request through the TrueFoundry AI Gateway (via the Playground or any API call).
  2. Log into your PromptLayer dashboard and click Traces in the left sidebar.
  3. Confirm that spans from tfy-llm-gateway are appearing with the full span hierarchy — including model calls, MCP gateway spans, and HTTP calls.
PromptLayer Traces page showing span hierarchy from TrueFoundry AI Gateway including AgentResponse, ChatCompletion, and Model spans with Success status
PromptLayer ingests spans asynchronously. Allow up to 60 seconds after your first gateway request before checking the Traces view.

​
Configuration Reference

ConfigurationValue
Traces Endpointhttps://api.promptlayer.com/v1/traces
Metrics EndpointNot supported by PromptLayer
ProtocolHTTP
EncodingProto
Auth Header KeyX-API-KEY
Auth Header ValueYour PromptLayer API key