> ## Documentation Index
> Fetch the complete documentation index at: https://www.truefoundry.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Fireworks AI Model Provider for TrueFoundry AI Gateway

> Add Fireworks AI as a model provider in the TrueFoundry AI Gateway and call its chat, completion and embedding models through one OpenAI-compatible API.

[Fireworks AI](https://fireworks.ai/) hosts open models behind an OpenAI-compatible API. TrueFoundry connects to Fireworks AI natively through the **AI Gateway**, so you can route to Fireworks models, set fallbacks and track cost the same way as for any other provider.

Fireworks AI supports these APIs through the gateway:

| API | Supported |
| - | - |
| Chat completions, including streaming, tool / function calling and JSON output | ✅ |
| Completions | ✅ |
| Embeddings | ✅ |
| Rerank, image generation, audio | Not available through this integration yet |

> [!NOTE]
>
> This page is about calling Fireworks AI models. To run a Fireworks-hosted safety model as a guardrail, see [Fireworks Safety Model Guardrail](/docs/ai-gateway/fireworks.md).

### Prerequisites

* A Fireworks AI API key. Create one in the Fireworks console.
* The model you want to call must be available to your Fireworks account. The model ID looks like `accounts/fireworks/models/kimi-k3` for a serverless model, or `accounts/<account>/deployments/<deployment>` for a dedicated deployment.

### Adding Models

This section explains how to add a Fireworks AI account and select the models you want to use.

**1. Open the provider list**

From the TrueFoundry dashboard, navigate to `AI Gateway` > `Models`, click `Add Model`, choose `To New Provider Account` and select `Fireworks AI`.

![Fireworks AI card in the Connect Models provider list](https://mintcdn.com/truefoundry/6viBBw1EfYsJNXEl/images/fireworks-ai-provider-list.jpg?fit=max\&auto=format\&n=6viBBw1EfYsJNXEl\&q=85\&s=90e2129b290e2a56ff79741b412074eb)

Select Fireworks AI from the provider list

**2. Add Fireworks AI account details**

Give a unique name to your Fireworks AI account (it defaults to `fireworks-ai`) and complete the form:

* **API Key**: your Fireworks AI API key. Store it as a TrueFoundry secret and select it here.
* **Base URL** (optional, under `Show advanced fields`): defaults to `https://api.fireworks.ai/inference/v1`. The URL must start with `https://` and cannot contain credentials, a query or a fragment. For Fireworks' US-only serverless endpoint set it to `https://us.api.fireworks.ai/inference/v1` and register the US model IDs, such as `accounts/fireworks/routers/kimi-k3-us`.
* **Discount %** (optional, under `Show advanced fields`): a discount from 0 to 100 applied to the upstream list price for this account.

![Fireworks AI account form with Name, API Key, Base URL and Discount fields](https://mintcdn.com/truefoundry/6viBBw1EfYsJNXEl/images/fireworks-ai-account-form.jpg?fit=max\&auto=format\&n=6viBBw1EfYsJNXEl\&q=85\&s=e6b48bfe03f3518bf29da21d0e0be3cb)

Fireworks AI account form with the advanced fields shown

Click `Continue to Models Selection`.

**3. Select models**

Tick the models you want to use. Chat models picked from this list take their price from the catalog, including the Fast and US-only routers, which carry their own prices. The list shows no price for embedding models and they are saved without a cost, so add the cost by hand if you want it tracked. The console names a picked model after its ID, for example `accounts-fireworks-models-kimi-k3`. Requests sent with `service_tier: priority` are billed by Fireworks above the standard rate but are still costed at the rate set on the model, so enter the priority rate through the pencil if you use it. Click the pencil next to a model to change its display name or cost before you save. To add a model that is not listed, such as a dedicated deployment, click `Add Model Manually`.

![Fireworks AI model catalog with input and output prices per million tokens](https://mintcdn.com/truefoundry/6viBBw1EfYsJNXEl/images/fireworks-ai-models-select.jpg?fit=max\&auto=format\&n=6viBBw1EfYsJNXEl\&q=85\&s=d8cd6e7ab61844a7e16d613d5de526d7)

Pick models from the catalog

For a manual model, set:

* **Display Name**: the name shown in the TrueFoundry UI and used in API calls (for example, `kimi-k3`).
* **Model ID**: the Fireworks model identifier (for example, `accounts/fireworks/models/kimi-k3`). Use the exact ID from the Fireworks model library or your deployment.
* **Model Types**: select **Chat**, **Completion** and **Embedding** as they apply to the model. Chat is the default.
* **Cost** (optional): turn it on to enter custom pricing per 1000 tokens (input, output, cache read and cache write), or per 1000 embedding tokens for embedding models. Fireworks bills Fast and US-only models, and requests sent with `service_tier: priority`, at higher rates than the standard serverless tier. Cost tracking uses the rates set on the model, so enter the matching rates for those models.

![Add new Model form with Display Name, Model ID, Model Types and Cost](https://mintcdn.com/truefoundry/6viBBw1EfYsJNXEl/images/fireworks-ai-model-form.jpg?fit=max\&auto=format\&n=6viBBw1EfYsJNXEl\&q=85\&s=3e1a405d56f0e2ac5784018f16fc5898)

Add a model by hand

**4. Save and check the models**

Click `Save without Testing`. The console confirms `models added successfully` and shows ready-to-run code samples. The models then appear in `AI Gateway` > `Models` under their display name, with the account name underneath.

![Models page listing a Fireworks AI chat and embedding model with cost per million tokens](https://mintcdn.com/truefoundry/6viBBw1EfYsJNXEl/images/fireworks-ai-registered-models.jpg?fit=max\&auto=format\&n=6viBBw1EfYsJNXEl\&q=85\&s=266f69e3a8ad5f4ae5643eeaa42479db)

Registered Fireworks AI models with their cost (example: display names shortened and costs entered by hand)

### Inference

After adding the models, call them through the AI Gateway like any other model, from the Playground or from your application with the OpenAI-compatible API. Use the TrueFoundry model ID in the format `your-fireworks-account/model-display-name`. A model picked from the catalog is named after its ID, for example `your-fireworks-account/accounts-fireworks-models-kimi-k3`, unless you renamed it.

```python lines theme={"dark"}
from openai import OpenAI

client = OpenAI(
    api_key="your_truefoundry_api_key",
    base_url="{GATEWAY_BASE_URL}"
)

response = client.chat.completions.create(
    model="my-fireworks-account/kimi-k3",  # provider account / model display name
    messages=[{"role": "user", "content": "Hello, how are you?"}]
)

print(response.choices[0].message.content)
```

For an embedding model, use the [embeddings API](/docs/ai-gateway/embed.md):

```python lines theme={"dark"}
embeddings = client.embeddings.create(
    model="my-fireworks-account/qwen3-embedding-8b",
    input=["hello", "world"]
)

print(len(embeddings.data[0].embedding))
```

### Request parameters

* The gateway sends the standard OpenAI chat parameters to Fireworks AI. Fireworks-specific parameters such as `top_k`, `reasoning_effort`, `prompt_cache_key` and `service_tier` are forwarded as you send them.
* `response_format` of type `json_object`, `json_schema` and Fireworks' own `grammar` is sent to Fireworks as is.
* Fireworks AI rejects fields it does not define with a `400` error ("Extra inputs are not permitted"). OpenAI-only fields such as `web_search_options` (chat) and `suffix` (completions) therefore return that error instead of being ignored.
* For reasoning models, the model's reasoning is returned in `reasoning_content` on the message, or on the streamed delta, and is counted in `completion_tokens_details.reasoning_tokens`.
* Fireworks AI caches repeated prompt prefixes on its side. Cached tokens are reported in `prompt_tokens_details.cached_tokens`.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.