This section explains the steps to add OpenRouter models and configure the required access controls.
1
Navigate to OpenRouter Models in AI Gateway
From the TrueFoundry dashboard, navigate to AI Gateway > Models and select OpenRouter.
Navigate to OpenRouter Models
2
Add OpenRouter Account Details
Click Add OpenRouter Account. Give a unique name to your OpenRouter account and complete the form with your OpenRouter authentication details (API Key). Add collaborators to your account, this will give access to the account to other users/teams. Learn more about access control here.
OpenRouter API Key Configuration
3
Add Models
To add a model, you need to get the model ID from OpenRouter:
Copy the model ID (e.g., anthropic/claude-3-haiku, openai/gpt-4o-mini)
Paste the model ID in the “Add Model” form in TrueFoundry AI Gateway
OpenRouter Models Page
Adding Model in TrueFoundry
You can add any model available on OpenRouter by copying its model ID from the OpenRouter website. TrueFoundry AI Gateway supports all text and multimodal models available through OpenRouter.
The complete list of models supported by OpenRouter can be found here. Make sure to use the exact model ID as shown on OpenRouter.
Once your OpenRouter model account is configured, the following API surfaces are available through the AI Gateway. The table below summarizes each endpoint alongside platform feature support (tracing, cost tracking).
OpenRouter models are added by pasting a model ID rather than picking from a catalog, so TrueFoundry does not ship public pricing for them. Set the per-token Cost on the model when you add it to get cost tracking and budget enforcement.
from openai import OpenAIclient = OpenAI( api_key="your-truefoundry-api-key", base_url="{GATEWAY_BASE_URL}",)response = client.chat.completions.create( model="openrouter-main/claude-3-haiku", messages=[{"role": "user", "content": "What is TrueFoundry in one line?"}],)print(response.choices[0].message.content)
Responses API
The AI Gateway forwards /responses requests straight to OpenRouter’s OpenAI-compatible Responses endpoint — the request is not translated into a chat completion, so reasoning, tool calling, and web search behave the same as they do when you call OpenRouter directly. Full docs: Responses API.
The Responses API requires the x-tfy-provider-name header. Set it on default_headers when you construct the client — the AI Gateway uses it to route the request to the right OpenRouter model account.
Python
from openai import OpenAIclient = OpenAI( api_key="your-truefoundry-api-key", base_url="{GATEWAY_BASE_URL}", default_headers={"x-tfy-provider-name": "openrouter-main"},)response = client.responses.create( model="openrouter-main/o4-mini", input=[{"role": "user", "content": "Give me a two-word tagline."}],)print(response.output_text)
Streaming
Set stream=True and iterate over the emitted events.
Python
stream = client.responses.create( model="openrouter-main/o4-mini", input=[{"role": "user", "content": "Tell me a short story."}], stream=True,)for event in stream: if event.type == "response.output_text.delta": print(event.delta, end="", flush=True)
OpenRouter’s Responses API is stateless. OpenRouter rejects requests that set store: true or a non-null previous_response_id with a 400, and it does not expose retrieve or delete for a response id — so client.responses.retrieve() and client.responses.delete() are not available for OpenRouter models. Carry the conversation yourself by appending each turn’s output to input on the next request.This also means an OpenRouter target cannot back a stateful Responses conversation on a virtual model. Send store=False on the first turn so the AI Gateway routes those requests statelessly.
