Skip to main content

Documentation Index

Fetch the complete documentation index at: https://www.truefoundry.com/llms.txt

Use this file to discover all available pages before exploring further.

Custom Endpoints let you register arbitrary HTTP services in the AI Gateway and route traffic through them. Unlike integrations that target a specific provider API (OpenAI, Anthropic, and so on), Custom Endpoints accept any HTTP upstream—internal REST APIs, Azure Speech Services, bespoke ML services, or third-party endpoints—while keeping a single entry point for your applications. Use Custom Endpoints when you want one TrueFoundry API key for callers, per-endpoint access control, and full tracing without distributing upstream credentials to every client. The gateway acts as a transparent proxy: it forwards the HTTP method, path, query string, headers, and body to your upstream base URL without translating payloads. Upstream credentials and extra headers are configured once on the provider account or integration; the gateway injects them on every upstream request automatically.

​
Setup

Custom Endpoints tab with endpoint groups and Add Custom Endpoint
1

Create a Custom Endpoint provider account

From the TrueFoundry dashboard, go to AI Gateway → Models, open the Custom Endpoints tab, and click Add Custom Endpoint.In the wizard, complete Configure Account: Name (required), optional Endpoint Type (None, Azure Speech Service, or Other), and optional Header Auth at the account level so integrations can inherit the same default upstream authentication. Use Continue to Endpoints when ready.
Setup Custom Endpoint Account wizard — Configure Account step
2

Add an integration (endpoint)

On the Endpoints wizard step, add or edit an endpoint integration (Add Endpoint from the list view works the same flow).Set Display Name and Base URL (the upstream origin — must not end with a trailing slash; see configuration reference). Optionally enable Custom Headers, Header Auth (per-endpoint upstream credentials), or TLS Settings.
Setup Custom Endpoint Account wizard — Endpoints step with integration fields
3

Make a request

Call the gateway using the pattern below. Replace GATEWAY_BASE_URL, providerAccountName, and endpointName with your values, and append any path that should be joined to the integration Base URL.
import requests

url = "{GATEWAY_BASE_URL}/proxy-api/{providerAccountName}/{endpointName}/your/upstream/path"

headers = {
    "Authorization": "Bearer your-truefoundry-api-key",
    "Content-Type": "application/json",
}

body = {"key": "value"}

response = requests.post(url, headers=headers, json=body)
print(response.status_code)
print(response.text)
Callers only need a TrueFoundry API key for the gateway. Upstream authentication is applied by the gateway from your provider account and integration settings—you do not pass upstream secrets from client code.
Usage code snippet modal showing Python Requests example for a custom endpoint
The wizard also includes an Access Control step after Endpoints (who can use this provider account), consistent with other AI Gateway providers — see Gateway access control.
The same manifest fields are available through the TrueFoundry CLI (tfy apply) if you configure provider accounts without the dashboard.

​
Endpoint structure

Requests use this URL shape (query parameters are forwarded unchanged): 
{GATEWAY_BASE_URL}/proxy-api/{providerAccountName}/{endpointName}/{upstream-path}?{query-params}
SegmentMeaning
providerAccountNameName of your Custom Endpoint provider account (lowercase letters, digits, hyphens; 3–32 characters)
endpointNameIntegration Display Name used in the URL (letters, digits, -, _, .; no spaces; 2–62 characters; cannot start with a digit)
upstream-pathPath appended to the integration’s Base URL
query-paramsOptional; passed through to the upstream as-is
Provider account and integration names must satisfy the patterns above when created. If your HTTP client requires encoding for certain characters in the path segment, URL-encode endpointName accordingly.

​
Authentication

Gateway authentication — Same as other AI Gateway routes: send Authorization: Bearer <TrueFoundry API key> (or your deployment’s documented gateway auth). Upstream authentication — Configure Header Auth (header name/value pairs used as upstream credentials) and optional Custom Headers on the provider account and/or each integration. The gateway adds these to the proxied request; callers never see upstream keys.
If an integration has no Header Auth, the gateway uses the provider account’s Header Auth when present. Setting Header Auth on an integration replaces the account default for that endpoint only.
Upstream services that expect Bearer or HTTP Basic credentials still use Header Auth: you store the exact header name and value the upstream expects (often Authorization). Examples: Bearer token
HeaderValue
AuthorizationBearer <your-upstream-access-token>
Enter the same Header name and value in the integration’s Header Auth (or account-level Header Auth). Username and password (HTTP Basic) Basic authentication sends a single Authorization header whose value is Basic followed by the Base64 encoding of username:password (UTF-8), with no newline inside that string.
  1. Build the string username:password (colon between user and password).
  2. Base64-encode it (standard Base64, padding as needed).
  3. Set Header Auth to Authorization = Basic <encoded-result>.
Example (compute once, paste the header value into the gateway): 
import base64

user = "myuser"
password = "mypass"
token = base64.b64encode(f"{user}:{password}".encode("utf-8")).decode("ascii")
# token is e.g. "bXl1c2VyOm15cGFzcw=="
print(f'Authorization: Basic {token}')
Equivalent one-liners: 
# macOS / Linux — prints the Base64 segment (prefix with `Basic ` for the full header value)
printf '%s' 'myuser:mypass' | base64
Paste Authorization = Basic <output-from-above> into Header Auth in the UI. API key in a custom header (e.g. Azure Speech)
HeaderValue
Ocp-Apim-Subscription-Key<subscription-key>

​
Use cases

import requests

# Integration Base URL should be like https://<region>.tts.speech.microsoft.com/cognitiveservices (no trailing slash).
# Append only the REST suffix — here `v1` for SSML — so the upstream becomes …/cognitiveservices/v1
url = "{GATEWAY_BASE_URL}/proxy-api/{providerAccountName}/{endpointName}/v1"

headers = {
    "Authorization": "Bearer your-truefoundry-api-key",
    "Content-Type": "application/ssml+xml",
    "X-Microsoft-OutputFormat": "audio-16khz-128kbitrate-mono-mp3",
}

ssml = """
<speak version='1.0' xml:lang='en-US'>
  <voice xml:lang='en-US' xml:gender='Female' name='en-US-JennyNeural'>
    Hello from the gateway.
  </voice>
</speak>
"""

response = requests.post(url, headers=headers, data=ssml.encode("utf-8"))
response.raise_for_status()
# Response body is binary audio from the upstream service.
open("output.mp3", "wb").write(response.content)

​
Configuration reference

​
Provider account (provider-account/custom-endpoint)

FieldDescription
NameUnique account identifier (lowercase letters, digits, hyphens; 3–32 characters); used in the gateway URL as providerAccountName
Endpoint TypeOptional: azure-speech-service or other (used for tracking / defaults in the product)
Header AuthOptional default upstream auth: header-based credentials (type: header plus a Headers map), applied when an integration has no Header Auth of its own
CollaboratorsWho can manage or use this account — see Gateway access control

​
Integration (endpoint)

FieldDescription
Display NameIdentifies the endpoint in the UI and in the URL as endpointName (pattern: letters, digits, -, _, .; no spaces; cannot start with a digit)
Base URLHTTPS (or HTTP) origin without a trailing slash; the gateway appends upstream-path from the request
Custom HeadersOptional key/value headers merged into every upstream request (often under Advanced)
Header AuthOptional upstream credentials as header key/value pairs; when set, replaces provider-account Header Auth for this integration
TLS SettingsOptional Reject Unauthorized toggle and optional Custom CA Certificates text for upstream TLS verification
Access ControlOptional authorized_subjects for per-subject allow lists — commonly edited via API or manifest rather than the default UI

​
Tracing

All traffic through Custom Endpoints is traced with span type CustomEndpoint on the gateway trace / root span. To browse requests in the dashboard, see View Traces. For general tracing concepts (traces, spans, attributes), see the LLM tracing overview.