> ## Documentation Index
> Fetch the complete documentation index at: https://www.truefoundry.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Structured Response Format
> Get consistent JSON outputs from chat completions API.
The chat completions API supports structured response formats, enabling you to receive consistent, predictable outputs in JSON format. This is useful for parsing responses programmatically.
## Overview
There are two primary approaches for structured responses:
1. **JSON Mode**: Basic JSON formatting without schema validation
2. **JSON Schema Mode**: Structured responses with strict schema validation and Pydantic integration
## Provider support for response schema
You can use `response_format` with any provider. The Gateway either uses the provider's native structured output or converts your schema into a tool the model must call and then puts the result in `message.content`.
| Provider | Support |
| ---------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- |
| **OpenAI** | Native for `json_object` and for `json_schema` on supported models (e.g. gpt-4o, gpt-5, gpt-4.1, o3, o4). Other models use tool conversion. |
| **Azure OpenAI** | Same as OpenAI. |
| **Anthropic** | Native for Claude 4.5/4.6 with `json_schema`. Other models use tool conversion. |
| **Google Gemini, Google Vertex** | Native when the request has no tools; otherwise tool conversion. |
| **All others** (Bedrock, Cohere, Mistral, OpenRouter, Groq, xAI, vLLM, etc.) | Tool conversion only. The Gateway turns your schema into a required tool and extracts the result into `message.content`. |
**Anthropic and JSON schema constraints:** The code examples in this doc use Pydantic's `ge=0` for fields such as `age`. Anthropic's API does not support these constraint parameters in the schema. If you use structured output with Anthropic models, omit `ge`, `le` and similar numeric/string constraints from your schema (or use a schema without them). The code will work with Anthropic once those constraints are removed.
## JSON Mode
JSON mode ensures the model's output is valid JSON without enforcing a specific structure:
```python lines theme={"dark"}
from openai import OpenAI
client = OpenAI(
api_key="your_truefoundry_api_key",
base_url="{GATEWAY_BASE_URL}"
)
response = client.chat.completions.create(
model="openai-main/gpt-4o",
messages=[
{"role": "system", "content": "You are a helpful assistant designed to output JSON."},
{"role": "user", "content": "Extract information about the 2020 World Series winner"}
],
response_format={"type": "json_object"}
)
print(response.choices[0].message.content)
```
**Output:**
```json lines theme={"dark"}
{
"team": "Los Angeles Dodgers",
"year": 2020,
"opponent": "Tampa Bay Rays",
"games_played": 6,
"series_result": "4-2"
}
```
## JSON Schema Mode
JSON Schema mode provides strict structure validation using predefined schemas:
```python lines theme={"dark"}
from openai import OpenAI
import json
client = OpenAI(
api_key="your_truefoundry_api_key",
base_url="{GATEWAY_BASE_URL}"
)
# Define JSON schema
user_info_schema = {
"type": "object",
"properties": {
"name": {"type": "string"},
"age": {"type": "integer", "minimum": 0},
"occupation": {"type": "string"},
"location": {"type": "string"},
"skills": {
"type": "array",
"items": {"type": "string"}
}
},
"required": ["name", "age", "occupation", "location", "skills"],
"additionalProperties": False
}
response = client.chat.completions.create(
model="openai-main/gpt-4o",
messages=[
{
"role": "system",
"content": "Extract user information and respond according to the provided JSON schema."
},
{
"role": "user",
"content": "My name is Sarah Johnson, I'm 28 years old, and I work as a data scientist in New York. I'm skilled in Python, SQL, and machine learning."
}
],
response_format={
"type": "json_schema",
"json_schema": {
"name": "user_info",
"schema": user_info_schema,
"strict": True
}
}
)
# Parse response
result = json.loads(response.choices[0].message.content)
```
**Output:**
```json lines theme={"dark"}
{
"name": "Sarah Johnson",
"age": 28,
"occupation": "data scientist",
"location": "New York",
"skills": ["Python", "SQL", "machine learning"]
}
```
When using JSON schema with strict mode set to true, all properties defined in the schema must be included in the required array. If any property is defined but not marked as required, the API will return a 400 Bad Request Error.
Pydantic provides automatic validation, serialization, and type hints for structured data:
```python lines theme={"dark"}
from openai import OpenAI
from pydantic import BaseModel, Field
from typing import List
client = OpenAI(
api_key="your_truefoundry_api_key",
base_url="{GATEWAY_BASE_URL}"
)
# Define Pydantic model
class UserInfo(BaseModel):
name: str = Field(description="Full name of the user")
age: int = Field(ge=0, description="Age in years")
occupation: str = Field(description="Job title or profession")
location: str = Field(description="City or location")
skills: List[str] = Field(description="List of professional skills")
class Config:
extra = "forbid" # Prevent additional fields
response = client.chat.completions.create(
model="openai-main/gpt-4o",
messages=[
{
"role": "system",
"content": "Extract user information and respond according to the provided schema."
},
{
"role": "user",
"content": "Hi, I'm Mike Chen, a 32-year-old software architect from Seattle. I specialize in cloud computing, microservices, and Kubernetes."
}
],
response_format={
"type": "json_schema",
"json_schema": {
"name": "user_info",
"schema": UserInfo.model_json_schema(),
"strict": True
}
}
)
# Parse and validate with Pydantic
user_data = UserInfo.model_validate_json(response.choices[0].message.content)
```
When using OpenAI models with Pydantic Models, there should not be any optional fields in the pydantic model when strict mode is true. This is because the corresponding JSON schema will have missing fields in the "required" section.
The beta parse client provides the most streamlined approach for Pydantic integration:
```python lines theme={"dark"}
from openai import OpenAI
from pydantic import BaseModel, Field
from typing import List, Optional
class UserInfo(BaseModel):
name: str = Field(description="Full name of the user")
age: int = Field(ge=0, description="Age in years")
occupation: str = Field(description="Job title or profession")
location: Optional[str] = Field(None, description="City or location")
skills: List[str] = Field(default=[], description="List of professional skills")
client = OpenAI(
api_key="your_truefoundry_api_key",
base_url="{GATEWAY_BASE_URL}"
)
completion = client.beta.chat.completions.parse(
model="openai-main/gpt-4o",
messages=[
{
"role": "system",
"content": "Extract user information from the provided text."
},
{
"role": "user",
"content": "Hello, I'm Alex Rodriguez, a 29-year-old product manager from Austin. I have experience in agile methodologies, data analysis, and team leadership."
}
],
response_format=UserInfo,
)
user_result = completion.choices[0].message.parsed
```
This approach allows for optional fields in your Pydantic model and provides a cleaner API for structured responses.