The Decisions API Is Now Live on TrueFoundry AI Gateway

The Decisions API Is Now Live on TrueFoundry AI Gateway

A lot of LLM calls in production exist to answer one question about a piece of text. Is this a refund request? How urgent is it? How frustrated is the customer? Decision models are built for that job. They answer typed questions and return probabilities instead of generated text: a yes/no probability, a pick from a fixed list, or a score on a scale. TrueFoundry AI Gateway now serves them on two routes, POST /v1/decisions and POST /v1/systemone, across seven providers.

You call them with the TrueFoundry API key you already use and address models as account-name/model-display-name. The gateway injects the provider credential. Tracing, cost tracking, rate limits, budgets and access control apply the same way they do for every other model on the gateway.

Try the Decisions API on the TrueFoundry AI Gateway →

What shipped

Two routes that share one provider list, so any model whose type is Decision works on either route:

• Two request formats. /v1/decisions (alias /decisions) takes OpenAI's Decisions format: an input plus a list of predicate, choice and score questions. /v1/systemone (alias /systemone) takes the System One format: a state plus a map of noul, choice and score questions, the same bodies TypeSafe Jev uses.
• Seven decision providers: OpenAI, TypeSafe (Jev, and Cloudflare Clef through Workers AI), OpenRouter, Perplexity, Databricks, Microsoft Foundry and self-hosted vLLM.
• Translation in both directions. The route you call does not limit which provider you can use. OpenAI receives OpenAI-format bodies, every other provider receives System One bodies, and answers come back in the format of the route you called.
• Gateway controls carry over: the same request logs and spans as other gateway APIs, cost tracking, rate limits, budgets, access control, and virtual models with load balancing and fallbacks.
• Input-only billing. Decision models bill input tokens only, so the output cost on the model is set to 0.

‍

One request body, any decision provider

The two formats describe the same idea in different words. In System One, a yes/no question is a noul and answers are keyed by the names you give your questions. In OpenAI's format it is a predicate, and answers come back as a list in question order. The gateway does the mapping per target. Text parts of an OpenAI input are joined into the System One state. A noul's criteria are folded into the instructions OpenAI sees. A System One question without instructions gets a generic one when it goes to OpenAI, which requires them.

The result is that switching provider is a change to the model string. Point a /v1/systemone body at an OpenAI decision model and you still get System One answers keyed by question name. Point a /v1/decisions body at a TypeSafe Jev model and you still get OpenAI-shaped answers.

Every answer carries probabilities. A score question with the levels calm, mildly annoyed and angry returns the score, a confidence and a probability per level, so your code can apply its own thresholds instead of parsing free text.

Why run decisions through the gateway

TypeSafe was already reachable through Custom Endpoints, which forward raw HTTP to the TypeSafe API. That path remains available as a transparent proxy, but it has no cost tracking, and you cannot switch providers by changing the model. The new routes use models in your catalog, so you get access control, tracing, cost tracking, virtual models, and the ability to change provider without changing the request body.

Virtual models are where this pays off. A virtual model whose model types include decision can load-balance or fail over across decision targets, including a mix of OpenAI and System One providers. The gateway translates each attempt into that target's format and translates the successful answer back into the format of the route you called. Your application calls one model name throughout.

How to turn it on

Step 1: Add the provider account and the model. In the gateway, open AI Gateway → Models, pick the provider and add its credentials. For TypeSafe that is a TypeSafe API key, with an optional base URL that points at Cloudflare Workers AI. Then add the model with Model Types set to Decision; a chat model cannot serve these routes. Pin a versioned model ID where the provider offers one (jev-1.13.0 rather than jev-latest) so confidence thresholds stay stable across releases, and set the output cost to 0.

Step 2: Set Access Control so the model is scoped to the right users and teams from the first request.

Step 3: Call either route. Send your TrueFoundry API key; the gateway injects the upstream credential. Set GATEWAY_BASE_URL without a trailing slash.

import os

import requests

gateway_base_url = os.environ["GATEWAY_BASE_URL"].rstrip("/")
response = requests.post(
    f"{gateway_base_url}/v1/systemone",
    headers={"Authorization": f"Bearer {os.environ['TFY_API_KEY']}"},
    json={
        "model": "my-typesafe-account/jev",
        "state": "Customer wrote: I was charged twice for my order #4411 and want one of the charges refunded today.",
        "questions": {
            "is_refund_request": {
                "type": "noul",
                "instructions": "Is the customer asking for a refund?",
            },
            "urgency": {
                "type": "choice",
                "instructions": "How urgent is this?",
                "criteria": {
                    "low": "can wait a week",
                    "high": "needs action today",
                },
            },
            "frustration": {
                "type": "score",
                "instructions": "How frustrated is the customer?",
                "criteria": ["calm", "mildly annoyed", "angry"],
            },
        },
    },
    timeout=60,
)
response.raise_for_status()
print(response.json()["answers"]["is_refund_request"])

Model IDs are namespaced as account-name/model-display-name. Change "model" to an OpenAI decision model such as openai-main/gpt-6-luna and OpenAI answers the same body, still keyed by question name. The OpenAI-format version of this request, and an image example, are in the Decisions API docs.

Before you move production traffic

• No streaming on either route.
• Image input works only with OpenAI models on /v1/decisions, as inline data: URLs; hosted URLs and file IDs are not supported. Other providers get a 400 before they are called. Cloudflare Clef, added through a TypeSafe account, accepts images on the System One images field instead.
• Self-hosted vLLM answers choice questions only, and needs a vLLM build that includes vllm#59299 (after v0.31.0).
• Some bodies are rejected early. Two OpenAI questions with the same name, a choice with both true and "true", or a one-label choice or one-level score sent to OpenAI all return a 400 before any provider is called.
• Refusals are typed. A question an OpenAI model will not answer comes back as {"type": "refusal", "name": ...}.
• Budgets, rate limits and access control already apply. Decision models are ordinary catalog models, so the per-team policy you have configured covers them.

Start today

Reach for a decision model when the output you need is a label, a yes/no or a score you will compare against a threshold: routing support tickets, flagging content, checking a document against a rule. Running it through TrueFoundry keeps the choice of provider in gateway config, where you can change it without touching the request body.

Try the Decisions API on the TrueFoundry AI Gateway →

Related reading

• Decisions API documentation (https://www.truefoundry.com/docs/ai-gateway/decisions): both routes, question types and translation limits
• TypeSafe on TrueFoundry AI Gateway (https://www.truefoundry.com/docs/ai-gateway/typesafe): add Jev, including through Cloudflare Workers AI
• Virtual models (https://www.truefoundry.com/docs/ai-gateway/virtual-model): load balancing and fallbacks
• Cost tracking (https://www.truefoundry.com/docs/ai-gateway/cost-tracking)
• What is an LLM Gateway? (https://www.truefoundry.com/blog/llm-gateway): the architectural primer

‍