> ## Documentation Index
> Fetch the complete documentation index at: https://www.truefoundry.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Messages API (/messages)

> Learn how to use Messages API through TrueFoundry's AI Gateway for interacting with Claude models.

**API Reference:** [`POST /messages`](/docs/api-reference/messages/messages)

## Provider capabilities

The table below summarizes gateway support for this endpoint by provider.

<Info>
  Legend:

  * **✅** Supported by provider and TrueFoundry
  * <Icon icon="circle-xmark" iconType="regular" color="red" /> Provided by provider, but not by TrueFoundry
  * <Icon icon="circle-minus" iconType="regular" /> Provider does not support this feature
</Info>

| Provider  | Messages API |
| --------- | ------------ |
| Anthropic | ✅            |

For every gateway endpoint and provider, see [Supported APIs](/docs/ai-gateway/intro-to-llm-gateway#supported-apis).

[Anthropic's Messages API](https://docs.anthropic.com/en/api/messages) is a powerful interface for interacting with Claude models. When using TruefFundry as your model gateway, you can access this API through a proxy endpoint that handles authentication and routing to the appropriate model.

## **Prerequisites**

To use the Anthropic Messages API through TrueFoundry, you'll need:

1. TrueFoundry API Key
2. Model account configured in TrueFoundry (Anthropic)
3. Python environment with `anthropic sdk` library installed

## Using the Anthropic SDK

The Anthropic Python SDK provides a convenient way to interact with Claude models. Here's how to configure it to work with the TrueFoundry proxy:

<Note>
  The AI Gateway accepts both Anthropic SDK auth patterns and translates internally:

  * `api_key=TFY_API_KEY` - SDK sends the `x-api-key` header
  * `auth_token=TFY_API_KEY` — SDK sends the `Authorization: Bearer` header

  Either works; the request body is identical. `api_key` is the idiomatic Anthropic SDK pattern - use it unless you have a reason to send a Bearer token.
</Note>

```python lines theme={"dark"}
from anthropic import Anthropic

BASE_URL = "{GATEWAY_BASE_URL}"
API_KEY = "your-truefoundry-api-key"

# Configure the Anthropic client to use TrueFoundry's Gateway
client = Anthropic(
    api_key=API_KEY,
    base_url=BASE_URL
)

# Make a request to the Messages API
def generate_response():
    response = client.messages.create(
        model="anthropic/claude-3-5",  # The model name configured in TrueFoundry
        max_tokens=1024,
        messages=[
            {
                "role": "user",
                "content": "Hello, Claude! Please explain quantum computing in simple terms."
            }
        ]
    )

    print(response.content)

generate_response()
```

## Request Format

When using the Messages API through TrueFoundry, your request should follow this format:

```json lines theme={"dark"}
{
  "model": "tfy-model-name",
  "max_tokens": 1024,
  "messages": [
    {
      "role": "user",
      "content": "Hello, Claude! Please explain quantum computing in simple terms."
    }
  ]
}
```

## Response Format

The response from the Messages API will have this structure:

```json lines theme={"dark"}
{
  "id": "msg_01XB89YSAA2VGMCF3ZS8ATTA1B",
  "type": "message",
  "role": "assistant",
  "content": [
    {
      "type": "text",
      "text": "Quantum computing is like traditional computing but it uses quantum bits or 'qubits' instead of regular bits. While traditional bits can only be in a state of 0 or 1, qubits can exist in multiple states simultaneously thanks to a quantum property called 'superposition.' This allows quantum computers to process certain types of information much faster than regular computers.\n\nAnother key quantum property is 'entanglement,' where qubits become connected and the state of one instantly affects the other, no matter the distance between them.\n\nThese properties give quantum computers the potential to solve certain complex problems much faster than traditional computers, like factoring large numbers (important for encryption) or simulating molecular structures (useful for drug development).\n\nHowever, quantum computers are still in early development stages. They're extremely sensitive to their environment and require special conditions like ultra-cold temperatures to operate. They're not replacements for regular computers but specialized tools for specific types of problems."
    }
  ],
  "model": "claude-3-opus-20240229",
  "stop_reason": "end_turn",
  "stop_sequence": null,
  "usage": {
    "input_tokens": 14,
    "output_tokens": 178
  }
}
```

## Prompt Caching

Prompt caching reuses previously computed prompt prefixes (a large `system` prompt, a shared context document, or a set of tool definitions) so the provider can skip reprocessing the cached portion on subsequent requests. This lowers both latency and cost.

There are three ways to cache with Claude models, and you can combine them:

* **Explicit caching** — you mark cache breakpoints yourself by adding `cache_control` to individual blocks (a `system` block, a message content block, or a `tool` definition). See [Anthropic's prompt caching guide](https://platform.claude.com/docs/en/build-with-claude/prompt-caching).
* **Automatic caching** — you add a single `cache_control` field at the **top level** of the request body, and the breakpoint is placed at the end of the prompt for you, so you don't manage breakpoints yourself.
* **Header-based caching** — you send the `x-tfy-cache-control` header and leave the body untouched. Equivalent to automatic caching, for clients that can't edit the request body. See [Caching with a header](#caching-with-a-header).

The AI Gateway forwards `cache_control` to native Claude providers **unchanged — it does not strip it** (the request is already in Anthropic's format). The AI Gateway only strips `cache_control` for providers that don't accept it, such as OpenAI or Gemini, which aren't served natively on the Messages endpoint (see the note below).

### Provider support

The Messages endpoint is served natively (in Anthropic's format) by Anthropic and by Claude models on Bedrock, Google Vertex, and Azure AI Foundry. All three ways to cache work on every one of them:

| Provider                  | Explicit (block-level) | Automatic (top-level `cache_control` or `x-tfy-cache-control`) |
| ------------------------- | ---------------------- | -------------------------------------------------------------- |
| Anthropic (direct)        | ✅                      | ✅                                                              |
| Claude Platform on AWS    | ✅                      | ✅                                                              |
| Azure AI Foundry (Claude) | ✅                      | ✅                                                              |
| AWS Bedrock (Claude)      | ✅                      | ✅                                                              |
| Google Vertex (Claude)    | ✅                      | ✅                                                              |

<Note>
  Only Anthropic, Claude Platform on AWS, and Azure AI Foundry read a top-level `cache_control` themselves — Bedrock and Google Vertex reject it. The AI Gateway papers over that difference by relocating a top-level `cache_control` onto the last cacheable block before forwarding, which every native Claude provider honours. The end result is the same either way: one breakpoint at the end of the prompt, caching the whole prefix.
</Note>

<Note>
  On Bedrock, prompt caching is **enabled by default** when the request goes through the InvokeModel API (which the Messages endpoint uses for Claude models). You can set explicit cache checkpoints at any point in your request body — across `system` blocks, message content blocks, and `tool` definitions — by attaching `cache_control` to each block you want to mark. Unlike the Converse-based `/chat/completions` path, these are forwarded as native Anthropic `cache_control` rather than rewritten into `cachePoint` markers.
</Note>

<Note>
  Non-Claude models (and provider-managed providers such as OpenAI or Gemini) are served on the Messages endpoint by translating the request into Chat Completions format. In that translation `cache_control` is dropped, but the provider's own prefix caching still applies transparently, and any cached-token counts are still reported in `usage`.
</Note>

### Explicit caching

Add `"cache_control": {"type": "ephemeral"}` to any `system` block, message content block, or `tool` definition you want cached. This works on every native Claude provider:

<CodeGroup>
  ```bash cURL theme={"dark"}
  curl -X POST "{GATEWAY_BASE_URL}/messages" \
    -H "Authorization: Bearer $TFY_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "model": "anthropic-main/claude-sonnet-4-20250514",
      "max_tokens": 1024,
      "system": [
        {
          "type": "text",
          "text": "<LONG_SYSTEM_PROMPT_OVER_THE_MIN_TOKENS>",
          "cache_control": { "type": "ephemeral" }
        }
      ],
      "messages": [
        { "role": "user", "content": "Summarize the key rules from the system prompt." }
      ]
    }'
  ```

  ```python Anthropic SDK theme={"dark"}
  from anthropic import Anthropic

  client = Anthropic(
      api_key="your-truefoundry-api-key",
      base_url="{GATEWAY_BASE_URL}",
  )

  message = client.messages.create(
      model="anthropic-main/claude-sonnet-4-20250514",
      max_tokens=1024,
      system=[
          {
              "type": "text",
              "text": "<LONG_SYSTEM_PROMPT_OVER_THE_MIN_TOKENS>",
              "cache_control": {"type": "ephemeral"},
          }
      ],
      messages=[
          {"role": "user", "content": "Summarize the key rules from the system prompt."}
      ],
  )

  print(message.usage)
  ```
</CodeGroup>

### Automatic caching

Add a single `cache_control` field at the **top level** of the request body instead of marking individual blocks. The breakpoint lands on the last cacheable block and advances as the conversation grows — useful for multi-turn chats. This works on every native Claude provider:

<CodeGroup>
  ```bash cURL theme={"dark"}
  curl -X POST "{GATEWAY_BASE_URL}/messages" \
    -H "Authorization: Bearer $TFY_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "model": "anthropic-main/claude-opus-4-8",
      "max_tokens": 1024,
      "cache_control": { "type": "ephemeral" },
      "system": "You are a helpful assistant that remembers our conversation.",
      "messages": [
        { "role": "user", "content": "My name is Alex. I work on machine learning." },
        { "role": "assistant", "content": "Nice to meet you, Alex!" },
        { "role": "user", "content": "What did I say I work on?" }
      ]
    }'
  ```

  ```python Anthropic SDK theme={"dark"}
  from anthropic import Anthropic

  client = Anthropic(
      api_key="your-truefoundry-api-key",
      base_url="{GATEWAY_BASE_URL}",
  )

  message = client.messages.create(
      model="anthropic-main/claude-opus-4-8",
      max_tokens=1024,
      cache_control={"type": "ephemeral"},
      system="You are a helpful assistant that remembers our conversation.",
      messages=[
          {"role": "user", "content": "My name is Alex. I work on machine learning."},
          {"role": "assistant", "content": "Nice to meet you, Alex!"},
          {"role": "user", "content": "What did I say I work on?"},
      ],
  )

  print(message.usage)
  ```
</CodeGroup>

### Caching with a header

When the request body is built by a framework or an off-the-shelf agent you can't modify, send the `x-tfy-cache-control` header instead. The AI Gateway injects the breakpoint at the end of the prompt for you, so the whole prefix — tool definitions, system prompt, and earlier turns — is cached:

<CodeGroup>
  ```bash cURL theme={"dark"}
  curl -X POST "{GATEWAY_BASE_URL}/messages" \
    -H "Authorization: Bearer $TFY_API_KEY" \
    -H "Content-Type: application/json" \
    -H "x-tfy-cache-control: {\"type\": \"ephemeral\"}" \
    -d '{
      "model": "anthropic-main/claude-sonnet-4-20250514",
      "max_tokens": 1024,
      "system": "<LONG_SYSTEM_PROMPT_OVER_THE_MIN_TOKENS>",
      "messages": [
        { "role": "user", "content": "Summarize the key rules from the system prompt." }
      ]
    }'
  ```

  ```python Anthropic SDK theme={"dark"}
  from anthropic import Anthropic

  client = Anthropic(
      api_key="your-truefoundry-api-key",
      base_url="{GATEWAY_BASE_URL}",
  )

  message = client.messages.create(
      model="anthropic-main/claude-sonnet-4-20250514",
      max_tokens=1024,
      system="<LONG_SYSTEM_PROMPT_OVER_THE_MIN_TOKENS>",
      messages=[
          {"role": "user", "content": "Summarize the key rules from the system prompt."}
      ],
      extra_headers={"x-tfy-cache-control": '{"type": "ephemeral", "ttl": "1h"}'},
  )

  print(message.usage)
  ```
</CodeGroup>

The header value is a JSON object with the same shape as the `cache_control` field you would have written in the body:

| Field  | Required | Description                                                                                                 |
| ------ | -------- | ----------------------------------------------------------------------------------------------------------- |
| `type` | Yes      | Cache type forwarded to the provider. Claude models accept `ephemeral`.                                     |
| `ttl`  | No       | Cache duration in Anthropic's format — `5m`, `1h`, `2d`. Ignored by providers that don't accept a duration. |

<Note>
  A `cache_control` in the body always wins — if the request already marks any block, the header is ignored, so hand-placed breakpoints are never overwritten. The value must be valid JSON and, if you set `ttl`, it must match the `5m` / `1h` / `2d` format, otherwise the AI Gateway rejects the request with a `400`.
</Note>

<Info>
  Don't confuse `x-tfy-cache-control` with [`x-tfy-cache-config`](/docs/ai-gateway/caching), which turns on the AI Gateway's own exact-match and semantic response cache. They work at different layers and can be used together.
</Info>

Anthropic enforces a minimum cacheable prefix length; shorter prompts accept the `cache_control` hint but are not actually cached:

| Minimum tokens | Models                                                         |
| -------------- | -------------------------------------------------------------- |
| `4096`         | Claude Mythos Preview, Opus 4.7, Opus 4.6, Opus 4.5, Haiku 4.5 |
| `2048`         | Sonnet 4.6, Haiku 3.5, Haiku 3                                 |
| `1024`         | Sonnet 4.5, Opus 4.1, Opus 4, Sonnet 4, Sonnet 3.7             |

### Cache usage in the response

When caching is active, the response `usage` object reports the cached token counts:

```json lines theme={"dark"}
{
  "usage": {
    "input_tokens": 120,
    "output_tokens": 210,
    "cache_creation_input_tokens": 4096,
    "cache_read_input_tokens": 0
  }
}
```

* `cache_creation_input_tokens`: tokens written to the cache (first call).
* `cache_read_input_tokens`: tokens served from the cache (subsequent calls).

## Advanced Features

The Messages API supports several advanced features:

### System Prompts

You can include a system prompt to guide Claude's behavior:

```python lines theme={"dark"}
client.messages.create(
    model="anthropic/claude-3-5",
    system="You are a helpful AI assistant that specializes in explaining complex topics simply.",
    messages=[
        {"role": "user", "content": "Explain quantum entanglement."}
    ]
)
```

### Multi-turn Conversations

For multi-turn conversations, include previous messages:

```python lines theme={"dark"}
client.messages.create(
    model="anthropic/claude-3-5",
    messages=[
        {"role": "user", "content": "What is machine learning?"},
        {"role": "assistant", "content": "Machine learning is a subset of artificial intelligence..."},
        {"role": "user", "content": "Can you explain supervised vs unsupervised learning?"}
    ]
)
```

### Streaming Responses

For streaming responses, use the streaming parameter:

```python lines theme={"dark"}
with client.messages.stream(
    model="anthropic/claude-3-5",
    messages=[{"role": "user", "content": "Write a short poem about AI."}]
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)

    # Access the final message at the end
    print("\nFinal message:", stream.get_final_message())
```