Baseten

Baseten hosts open-source and custom models behind an OpenAI-compatible Model API. TrueFoundry connects to Baseten through the AI Gateway at https://inference.baseten.co/v1. Baseten models support chat completions, streaming, and embeddings (where the model supports it).

Adding Models

This section explains the steps to add Baseten models and configure the required access controls.

Navigate to Baseten in AI Gateway

From the TrueFoundry dashboard, navigate to AI Gateway > Models and select Baseten.

Add Baseten Account Details

Click Add Baseten Account. Give a unique name to your Baseten account and complete the form with your Baseten authentication details (API Key). Add collaborators to your account, this will give access to the account to other users/teams. Learn more about access control here.Under advanced settings, you can optionally set a Discount % that is applied to the upstream list price for this account.

Add Models

Click + Add Model and fill the form:

Display Name: the name shown in the TrueFoundry UI.
Model ID: the model identifier from Baseten’s model catalog.
Model Types: select the type of the model (defaults to Chat).
Cost (optional): cost metrics used for cost tracking.

Inference

After adding the models, you can perform inference using an OpenAI-compatible API via the Playground or by integrating with your own application. Use the TrueFoundry model ID in the format your-baseten-account/your-model-display-name. See Chat Completions - Getting Started for a full code example:

from openai import OpenAI

client = OpenAI(
    api_key="your_truefoundry_api_key",
    base_url="{GATEWAY_BASE_URL}"
)

response = client.chat.completions.create(
    model="my-baseten-account/my-model",  # provider account / model display name
    messages=[{"role": "user", "content": "Hello, how are you?"}]
)

print(response.choices[0].message.content)

Get Started

LLM Gateway

MCP Registry and Gateway

Skills Registry

Prompt Registry

Guardrails and Security

Observability

Deployment

Admin Guide

Chat

Messages

Embeddings

Rerank

Responses

Image

Audio

Batch

Files

Fine-tuning

Moderations

Video

Models

Adding Models

Inference

​Adding Models

​Inference

Adding Models

Inference