Guardrails AI Integration

Documentation Index

Fetch the complete documentation index at: https://www.truefoundry.com/llms.txt

Use this file to discover all available pages before exploring further.

​

What is Guardrails AI?

​

Key Features of Guardrails AI on TrueFoundry

1. PII detection (email, phone, SSN, credit card, IBAN, IP, passport, driver license) on inbound user messages and outbound assistant responses via DetectPII.
2. Secrets detection (AWS keys, OpenAI tokens, GitHub tokens, JWT, private keys) via SecretsPresent.
3. Toxic-language detection via the Unitary classifier (ToxicLanguage).
4. Profanity filter on assistant output via ProfanityFree.
5. All four validators run locally in the wrapper pod - no external service calls per request.

​

Architecture

• {"verdict": true} - allow
• {"verdict": false, "message": "..."} - block

​

Prerequisites

• A TrueFoundry workspace you can deploy services into.
• A Guardrails Hub API token from hub.guardrailsai.com/keys. The free tier is sufficient.
• The model FQN you want to protect (e.g. openai-main/gpt-4o-mini).
• A cluster with a configured base host (visible at Integrations → Clusters → <cluster>).

​

Integration Steps

1

Clone the wrapper repository

Clone the integration repo and switch to the Guardrails AI folder:

git clone https://github.com/truefoundry/integrations-custom-guardrails
cd integrations-custom-guardrails/integrations/guardrails-ai

2

Configure environment variables

Copy .env.example to .env and fill in the values. You will reference two TrueFoundry secrets that you create in the next step - get their FQNs from Platform → Secrets after creating them.

.env

# Runtime + build-time tokens
GUARDRAILS_TOKEN=<your Hub API token>
WRAPPER_API_KEY=<generate with `python -c "import secrets; print(secrets.token_urlsafe(32))"`>

# Deploy-time only
TFY_WORKSPACE_FQN=<cluster>:<workspace>
TFY_PUBLIC_HOST=ml.<cluster>.truefoundry.cloud
TFY_PUBLIC_PATH=/guardrails-ai-tfy

WRAPPER_API_KEY_SECRET_FQN=tfy-secret://<workspace>/guardrails-ai-tfy/wrapper-api-key
GUARDRAILS_TOKEN_SECRET_FQN=tfy-secret://<workspace>/guardrails-ai-tfy/guardrails-token

Generate WRAPPER_API_KEY with python -c "import secrets; print(secrets.token_urlsafe(32))". The gateway will send this value as Authorization: Bearer … when calling the wrapper.

3

Create two TrueFoundry secrets

Navigate to Platform → Secrets and create a Secret Group named guardrails-ai-tfy with two secrets:

Secret Name	Value
guardrails-token	Your Hub API token. Consumed at Docker build time to install validators.
wrapper-api-key	The same random string you put in .env as WRAPPER_API_KEY.

Copy each secret’s FQN and confirm the entries in .env (WRAPPER_API_KEY_SECRET_FQN, GUARDRAILS_TOKEN_SECRET_FQN) match.

4

Deploy the wrapper service

Install the TrueFoundry CLI, log in, and deploy:

pip install -U truefoundry
tfy login
python deploy.py --wait

The first build is slow (~5 min) because the Dockerfile pulls HuggingFace classifier weights for ToxicLanguage at build time. Subsequent builds use TrueFoundry’s image layer cache and are much faster. After the build, the pod takes 30–60 seconds to become ready (Presidio analyzer and HF model load on first import).

Verify the service is healthy:

curl -s https://ml.<cluster>.truefoundry.cloud/guardrails-ai-tfy/health
# {"status":"ok"}

5

Register the Custom Guardrail Configs in TrueFoundry

Navigate to AI Gateway → Guardrails → + Add New Guardrails Group.

1. Group name: guardrails-ai
2. Description (optional): Guardrails AI Hub: PII, secrets, toxicity, profanity
3. Click + Add Guardrail Config → Custom Guardrail Config seven times - one per guardrail. Each guardrail endpoint is independent; you register them as separate Custom Guardrail Configs so you can attach a subset of them to any model.

For each guardrail, use the same template:

Field	Value
Name	guardrails-ai-<validator>-<direction> (e.g. guardrails-ai-detect-pii-input)
Operation	Validate
URL	https://ml.<cluster>.truefoundry.cloud/guardrails-ai-tfy/<validator>-<direction>
Auth Data	Custom Bearer Auth, token = the wrapper-api-key secret value
Headers	(empty)
Config	{}
Enforcing Strategy	Enforce But Ignore On Error (recommended)

The seven guardrails to register:

Validator	Direction	Name	URL suffix
DetectPII	Input Guardrail	guardrails-ai-detect-pii-input	/detect-pii-input
DetectPII	Output Guardrail	guardrails-ai-detect-pii-output	/detect-pii-output
SecretsPresent	Input Guardrail	guardrails-ai-secrets-present-input	/secrets-present-input
SecretsPresent	Output Guardrail	guardrails-ai-secrets-present-output	/secrets-present-output
ToxicLanguage	Input Guardrail	guardrails-ai-toxic-language-input	/toxic-language-input
ToxicLanguage	Output Guardrail	guardrails-ai-toxic-language-output	/toxic-language-output
ProfanityFree	Output Guardrail	guardrails-ai-profanity-free-output	/profanity-free-output

Save the group.

The wrapper signals guardrail decisions via {"verdict": true \| false} on HTTP 200 - real failures (validator load error, wrapper crash) come as HTTP 5xx. With Enforce But Ignore On Error, transient outages pass through while real policy decisions still block. Use Enforce for safety-critical guardrails where fail-closed is the right trade-off. See Custom guardrail response contract and Enforcing Strategy.

6

Apply the guardrail to traffic

There are two ways to route requests through the rails - pick based on whether you want every call to a model protected, or per-call opt-in.

• Pin to a model (every call protected)
• Per-request opt-in

Navigate to AI Gateway → Models → <model> → Guardrails tab → attach the guardrails-ai group → Save. Every caller of this model now passes through the rails.

Send the X-TFY-GUARDRAILS header on individual requests, listing the per-rail selectors you want active. Selector format is <group-name>/<config-name>; omit selectors for any rails you don’t want active on a given request.

from openai import OpenAI
import json

client = OpenAI(
    api_key="<TFY API key>",
    base_url="https://gateway.truefoundry.ai",
)

resp = client.chat.completions.create(
    model="openai-main/gpt-4o-mini",
    messages=[{"role": "user", "content": "What is the capital of France?"}],
    extra_headers={
        "X-TFY-GUARDRAILS": json.dumps({
            "llm_input_guardrails": [
                "guardrails-ai/guardrails-ai-detect-pii-input",
                "guardrails-ai/guardrails-ai-secrets-present-input",
                "guardrails-ai/guardrails-ai-toxic-language-input",
            ],
            "llm_output_guardrails": [
                "guardrails-ai/guardrails-ai-detect-pii-output",
                "guardrails-ai/guardrails-ai-secrets-present-output",
                "guardrails-ai/guardrails-ai-toxic-language-output",
                "guardrails-ai/guardrails-ai-profanity-free-output",
            ],
        }),
    },
)

7

Test end-to-end

Issue two test calls through the gateway - one that should succeed and one that should be blocked:

GW=https://gateway.truefoundry.ai
TFY_KEY=<your TFY API key>
MODEL=openai-main/gpt-4o-mini

# Should succeed with a normal completion
curl -s "$GW/chat/completions" \
  -H "Authorization: Bearer $TFY_KEY" -H "Content-Type: application/json" \
  -H 'X-TFY-GUARDRAILS: {"llm_input_guardrails":["guardrails-ai/guardrails-ai-detect-pii-input"],"llm_output_guardrails":["guardrails-ai/guardrails-ai-detect-pii-output"]}' \
  -d "{\"model\":\"$MODEL\",\"messages\":[{\"role\":\"user\",\"content\":\"What is the capital of France?\"}]}"

# Should be blocked: guardrail_checks_failed (PII detected)
curl -s "$GW/chat/completions" \
  -H "Authorization: Bearer $TFY_KEY" -H "Content-Type: application/json" \
  -H 'X-TFY-GUARDRAILS: {"llm_input_guardrails":["guardrails-ai/guardrails-ai-detect-pii-input"]}' \
  -d "{\"model\":\"$MODEL\",\"messages\":[{\"role\":\"user\",\"content\":\"My email is jane.doe@example.com and my SSN is 123-45-6789\"}]}"

A successful block returns:

{
  "status": "failure",
  "message": "Input Guardrail checks failed for integrations: [guardrails-ai/guardrails-ai-detect-pii-input] ...",
  "error": { "type": "guardrail_checks_failed", "code": "400" },
  "guardrail_checks": {
    "input_guardrails": [{
      "guardrail_integration": "guardrails-ai/guardrails-ai-detect-pii-input",
      "result": "failed",
      "data": {
        "verdict": false,
        "explanation": "DetectPII (input): Validation failed for field with errors: ...",
        "guardrailUrl": "https://..."
      }
    }]
  }
}

The blocking validator’s message is preserved in guardrail_checks.input_guardrails[0].data.explanation.

​

Customizing the Validator Bundle

​

Adding a new validator

from guardrail.restrict_to_topic_input import restrict_to_topic_input

RAIL_ROUTES["/restrict-to-topic-input"] = restrict_to_topic_input

python deploy.py --wait

​

Useful Hub validators

​

Troubleshooting

curl -sS -X POST https://ml.<cluster>.truefoundry.cloud/guardrails-ai-tfy/detect-pii-input \
  -H "Authorization: Bearer $WRAPPER_API_KEY" -H "Content-Type: application/json" \
  -d '{"requestBody":{"messages":[{"role":"user","content":"<your test prompt>"}]},"context":{"user":{"subjectId":"u1","subjectType":"user"}}}'

curl -sS https://ml.<cluster>.truefoundry.cloud/guardrails-ai-tfy/debug/loaded-config \
  -H "Authorization: Bearer $WRAPPER_API_KEY" | jq

guardrails-ai @ git+https://github.com/guardrails-ai/guardrails.git@v0.9.3

​

Known Limitations

• Validator accuracy is context-sensitive. See troubleshooting above. v1 is “defense in depth, not perfect prevention.” Layer with your application’s own checks.
• No streaming-aware guardrails. The TrueFoundry custom-guardrail contract is buffered: the gateway holds the full assistant response before calling the output rail. Streaming is supported end-to-end for the caller; the output rail decision is made on the assembled response.
• No mutation mode. All v1 validators run in on_fail="exception". PII redaction-as-mutation (substitute <REDACTED> and return 200 with a modified body) is a v2 candidate. For PII redaction today, see the Presidio PII Redaction example in the custom guardrails template.
• Validator versions pin at build time. Hub validator updates require a wrapper rebuild + redeploy.
• In-memory state is per-replica. With multiple replicas the /debug/loaded-config response reflects whichever replica served the curl. After a deploy, retry the curl 5–10 times to surface heterogeneity.

​

Reference