Fireworks AI Model Provider for TrueFoundry AI Gateway

​

Prerequisites

• A Fireworks AI API key. Create one in the Fireworks console.
• The model you want to call must be available to your Fireworks account. The model ID looks like accounts/fireworks/models/kimi-k3 for a serverless model, or accounts/<account>/deployments/<deployment> for a dedicated deployment.

​

Adding Models

1

Open the provider list

From the TrueFoundry dashboard, navigate to AI Gateway > Models, click Add Model, choose To New Provider Account and select Fireworks AI.

2

Add Fireworks AI account details

Give a unique name to your Fireworks AI account (it defaults to fireworks-ai) and complete the form:

• API Key: your Fireworks AI API key. Store it as a TrueFoundry secret and select it here.
• Base URL (optional, under Show advanced fields): defaults to https://api.fireworks.ai/inference/v1. The URL must start with https:// and cannot contain credentials, a query or a fragment. For Fireworks’ US-only serverless endpoint set it to https://us.api.fireworks.ai/inference/v1 and register the US model IDs, such as accounts/fireworks/routers/kimi-k3-us.
• Discount % (optional, under Show advanced fields): a discount from 0 to 100 applied to the upstream list price for this account.

Click Continue to Models Selection.

3

Select models

Tick the models you want to use. Chat models picked from this list take their price from the catalog, including the Fast and US-only routers, which carry their own prices. The list shows no price for embedding models and they are saved without a cost, so add the cost by hand if you want it tracked. The console names a picked model after its ID, for example accounts-fireworks-models-kimi-k3. Requests sent with service_tier: priority are billed by Fireworks above the standard rate but are still costed at the rate set on the model, so enter the priority rate through the pencil if you use it. Click the pencil next to a model to change its display name or cost before you save. To add a model that is not listed, such as a dedicated deployment, click Add Model Manually.

For a manual model, set:

• Display Name: the name shown in the TrueFoundry UI and used in API calls (for example, kimi-k3).
• Model ID: the Fireworks model identifier (for example, accounts/fireworks/models/kimi-k3). Use the exact ID from the Fireworks model library or your deployment.
• Model Types: select Chat, Completion and Embedding as they apply to the model. Chat is the default.
• Cost (optional): turn it on to enter custom pricing per 1000 tokens (input, output, cache read and cache write), or per 1000 embedding tokens for embedding models. Fireworks bills Fast and US-only models, and requests sent with service_tier: priority, at higher rates than the standard serverless tier. Cost tracking uses the rates set on the model, so enter the matching rates for those models.

4

Save and check the models

Click Save without Testing. The console confirms models added successfully and shows ready-to-run code samples. The models then appear in AI Gateway > Models under their display name, with the account name underneath.

​

Inference

from openai import OpenAI

client = OpenAI(
    api_key="your_truefoundry_api_key",
    base_url="{GATEWAY_BASE_URL}"
)

response = client.chat.completions.create(
    model="my-fireworks-account/kimi-k3",  # provider account / model display name
    messages=[{"role": "user", "content": "Hello, how are you?"}]
)

print(response.choices[0].message.content)

embeddings = client.embeddings.create(
    model="my-fireworks-account/qwen3-embedding-8b",
    input=["hello", "world"]
)

print(len(embeddings.data[0].embedding))

​

Request parameters

• The gateway sends the standard OpenAI chat parameters to Fireworks AI. Fireworks-specific parameters such as top_k, reasoning_effort, prompt_cache_key and service_tier are forwarded as you send them.
• response_format of type json_object, json_schema and Fireworks’ own grammar is sent to Fireworks as is.
• Fireworks AI rejects fields it does not define with a 400 error (“Extra inputs are not permitted”). OpenAI-only fields such as web_search_options (chat) and suffix (completions) therefore return that error instead of being ignored.
• For reasoning models, the model’s reasoning is returned in reasoning_content on the message, or on the streamed delta, and is counted in completion_tokens_details.reasoning_tokens.
• Fireworks AI caches repeated prompt prefixes on its side. Cached tokens are reported in prompt_tokens_details.cached_tokens.