Onyx runtime protection now on the TrueFoundry AI Gateway: policy guardrails for every LLM call

If your team routes LLM traffic through an AI gateway, every prompt and every response is a point where something can go wrong. The TrueFoundry AI Gateway gives you unified access to more than 1,000 LLMs behind a single OpenAI-compatible API. Onyx is now available as a custom guardrail on that gateway, so TrueFoundry customers can add runtime protection without changing application code.

Runtime protection belongs at the enforcement point, not in each application. Onyx runs the same checks on every prompt and response that crosses that boundary. Your applications get Onyx's prompt-injection defense, jailbreak detection, data-exfiltration checks, and content policies evaluated before the model or the user ever sees the output.

Setup is configuration only. There is no wrapper to deploy and no application code to change.

Prerequisites

• A TrueFoundry AI Gateway with at least one model provider configured, that honors verdict: false on HTTP 200.
• An Onyx AI Guard policy and its Guard Token, with at least one Block rule scanning the Input and/or Output direction you want enforced.
• Outbound HTTPS from your TrueFoundry gateway to your Onyx tenant host.
• Permission to add and attach guardrail groups in the TrueFoundry dashboard.

Step-by-step integration

Onyx exposes a TrueFoundry-native evaluate endpoint that speaks the gateway's custom-guardrail contract directly. TrueFoundry calls Onyx, Onyx returns a verdict. No wrapper or adapter sits in between.

Step 1: Get your Guard Token and build the URL

The Guard Token in your AI Guard policy URL identifies which policy to evaluate, and it is the auth for the call. It is not an API key or MCP Gateway Token. In Onyx, go to Policies, then Runtime Policies, then the AI Guard tab, open the policy's AI Guard URL, and copy the Guard Token and your tenant hostname.

Build the guardrail URL:

https://<routing-id>.ai-guard.onyx.security/guard/evaluate/v1/<guard-token>/truefoundry

Use your own tenant host. The bare https://ai-guard.onyx.security does not route to a tenant and returns 404.

Step 2: Register the custom guardrail configs

In the TrueFoundry dashboard, go to AI Gateway, then Guardrails, then New Guardrails Group. Choose Custom and name it onyx-security. Add two configs, one per direction, both pointing at the same Onyx URL.

For the input config, set the name to onyx-input, the operation to Validate, and the target to Request. Paste the guardrail URL you built in Step 1, leave Auth Data as None, set Config to {}, and choose Enforce But Ignore On Error as the enforcing strategy.

For the output config, set the name to onyx-output, the operation to Validate, and the target to Response. Use the same guardrail URL, leave Auth Data as None, set Config to {}, and choose Enforce But Ignore On Error as the enforcing strategy.

The only differences between the two configs are the name and the target. Leave Auth Data empty. The Guard Token in the URL path is the auth, so no Bearer token is required.

Step 3: Attach the guardrail to traffic

Attach the group to a model in the model's guardrail settings, apply it across the gateway through Policies then Guardrails, or select it per request with a header so you can test without changing a model:

{

  "llm_input_guardrails": ["onyx-security/onyx-input"],

  "llm_output_guardrails": ["onyx-security/onyx-output"]

}

Send a benign prompt and it passes through to the model. Send one that trips an Onyx rule and the gateway blocks it, returning the policy's message instead of a model answer, as HTTP 400 with error.type: guardrail_checks_failed. For response guarding, set "stream": false; output rails do not run on streamed responses.

What you unlock

Input and output validation. The input rail screens prompts before the model runs. The output rail screens responses before they reach the user, on non-streaming requests. Onyx evaluates each against your policy's Input-direction and Output-direction rules.

Policy-driven blocking. Blocks carry the message you configured in Onyx, so the caller sees your wording, not a generic error. You change enforcement by editing the policy in the Onyx console, with no changes on the TrueFoundry side.

One policy across every application. Because the check runs at the gateway, every team and every app that routes through it inherits the same guardrails. There is no per-application integration to maintain and no coverage gap when a new service ships.

Observability in the same place. Guardrail decisions are traced alongside the rest of the request. The gateway is OpenTelemetry-compliant, so you can see which requests were blocked and why in the same stack you already use for latency and cost.

FAQ

What are LLM guardrails on an AI gateway?

Guardrails are checks the gateway runs on prompts and responses before they reach the model or the user. On the TrueFoundry AI Gateway they run as configurable policies, so you can block prompt injection, jailbreaks, or data leakage centrally rather than in each application.

Do I have to deploy anything to add the Onyx guardrail?

No. TrueFoundry calls the Onyx evaluate endpoint directly. You register two guardrail configs that point at your Onyx URL. There is no wrapper or adapter to host.

Does the Onyx guardrail run on both prompts and responses?

Yes. The integration registers two rails, one on the request and one on the response. Whether a phrase is blocked on output depends on your Onyx policy having an Output-direction rule; Input and Output are configured separately in Onyx, and output guarding requires non-streaming requests.

Do I have to change my application code to add guardrails?

No. The guardrail runs at the gateway. You attach it to a model or pass a header on the request, and your application code keeps working.

Why is my request returning a validation error?

The /truefoundry endpoint requires context.user.subjectSlug on the request. The gateway supplies this on normal traffic; if you call the endpoint directly for testing, include subjectSlug or Onyx returns a block verdict with "could not validate the request."

Can I run the TrueFoundry AI Gateway in my own VPC?

Yes. TrueFoundry supports VPC deployment, on-prem, air-gapped, and multi-cloud setups, with no data leaving your domain.

Does it integrate with my existing observability stack?

Yes. The gateway is OpenTelemetry-compliant and plugs into Grafana, Datadog, or Prometheus, tracing each request from prompt to model, guardrail decisions included.

Onyx and TrueFoundry collaborated on this integration. Reach out to your TrueFoundry or Onyx representative to get started.