> ## Documentation Index
> Fetch the complete documentation index at: https://www.truefoundry.com/llms.txt
> Use this file to discover all available pages before exploring further.

# AI Gateway Guardrails Overview

> Learn about guardrails for ensuring safety, compliance, and quality in LLM interactions and MCP tool invocations.

## Why Guardrails?

Once AI applications go to production, they handle real user data and — in the case of agents — call external tools on their own. Things can go wrong fast:

* **A customer support chatbot** leaks a user's credit card number because PII wasn't stripped from the context.
* **A coding agent** runs `rm -rf /` through an MCP tool after hallucinating a shell command — and nothing stopped it.
* **A healthcare assistant** makes up drug dosage numbers. The response reaches the patient unchecked.
* **An internal Q\&A bot** gets jailbroken through prompt injection, leaking confidential company data.

Guardrails prevent these scenarios. They sit between your application and the LLM (or MCP tool), inspecting and — when needed — blocking or rewriting data before it causes damage. You can attach them to **LLM requests** (check the prompt going in, check the response coming out) and to **MCP tool calls** (check the arguments before the tool runs, check the results after it returns).

## How a TrueFoundry Guardrail Works

Each guardrail has two settings you configure: what it *does* with the data, and how strictly it *enforces* its decisions.

### Operation Mode

| Mode | Behavior | Execution |
| - | - | - |
| **Validate** | Looks at the data, blocks the request if something is wrong. Doesn't touch the data itself.  `E.g., a content moderation guardrail sees hate speech in the prompt and blocks the request outright.` | **LLM Input Validation** can run **in parallel with the model request** (see below). **LLM Output Validation** and **MCP Pre/Post Tool** validation run **synchronously** in the request path before the response or tool result is released. |
| **Mutate** | Looks at the data *and* rewrites it. Can also block.  `E.g., a PII guardrail rewrites `"My SSN is 123-45-6789"`to`"My SSN is REDACTED"` and lets the request through.` | Runs **sequentially** by priority (lower = first) |

> [!NOTE]
>
> **Request Traces vs. this table:** Traces show each guardrail as its own span with start/end times. **LLM Input Validation** spans often overlap the model span because validation runs alongside the in-flight model request. **Output** and **MCP** guardrail spans typically appear **after** the model or tool span finishes — that ordering reflects synchronous evaluation, not a contradiction with the execution model above.

### Enforcement Strategy

This decides what happens when a guardrail catches a violation — and also what happens if the guardrail *itself* has a problem (like a timeout or a provider outage).

| Strategy | On Violation | On Guardrail Error |
| - | - | - |
| **Enforce** | Block | Block |
| **Enforce But Ignore On Error** | Block | Let through (graceful degradation) |
| **Audit** | Let through (log only) | Let through |

> [!TIP]
>
> **How to roll out safely**:
>
> * Start with **Audit** so you can see what guardrails would catch without affecting users.
> * Once things look right, switch to **Enforce But Ignore On Error** — you get protection, but a guardrail provider outage won't take your app down.
> * Move to **Enforce** when you need strict compliance.

## How TrueFoundry AI Gateway Runs Guardrails

Where guardrails run depends on whether you're making an LLM call or invoking an MCP tool.

> [tab] LLM Requests
>
> LLM requests have two hooks — **Input** (before the model sees the prompt) and **Output** (after the model responds):
>
> > [card] LLM Input
> >
> > Runs **before** the prompt reaches the LLM:
> >
> > * PII masking and redaction
> > * Prompt injection detection
> > * Content moderation
>
> > [card] LLM Output
> >
> > Runs **after** the LLM responds:
> >
> > * Hallucination detection
> > * Secrets detection
> > * Content filtering
>
> ![Diagram showing the flow of LLM requests through input and output guardrails](https://mintcdn.com/truefoundry/Fc6z3zZxeYI2aatT/images/llm_guardrails_flow.png?fit=max\&auto=format\&n=Fc6z3zZxeYI2aatT\&q=85\&s=ae40c961eed9a85488d594f74528777a)
>
> LLM Request Guardrail Flow
>
> Here's the order of operations when a request hits the AI Gateway:
>
> 1. **Input Mutation** guardrails run first and block until they finish (e.g., redacting PII from the prompt).
> 2. **Input Validation** kicks off in the background — it checks for things like prompt injection *while* the model request is already in flight.
> 3. The **model request starts** with the mutated prompt.
> 4. If **Input Validation fails** while the model is still running, the AI Gateway **cancels the model request** right away so you don't pay for it.
> 5. Once the model responds, **Output Mutation** guardrails process the response (e.g., stripping secrets).
> 6. **Output Validation** checks the final result. If it fails, the response is blocked — though model costs have already been incurred at this point.
> 7. The clean response goes back to the client.
>
> | Hook | Execution | What Happens on Failure |
> | - | - | - |
> | **Input Validation** | Async (parallel with model request) | Model request cancelled |
> | **Input Mutation** | Sync (before model request) | Request blocked |
> | **Output Mutation** | Sync (after model response) | Response blocked |
> | **Output Validation** | Sync (after output mutation) | Response rejected |
>
> > [!WARNING]
> >
> > **Streaming (`stream: true`) and LLM output guardrails:** Output guardrails are **not applied** when the response is streamed (`"stream": true`). This is because output guardrails need the complete response text to evaluate, but streaming sends the response in chunks as they are generated. If you need output guardrails to run, set `"stream": false` in your request. Input guardrails work the same way regardless of streaming mode — they always run before the request is sent to the model. Guardrails are also not applied on text-to-speech, speech-to-text, or realtime requests. See the [FAQ](#faq) for more details.

> [tab] MCP Tool Invocations
>
> MCP tool calls have two hooks — **Pre Tool** (before the tool runs) and **Post Tool** (after it returns):
>
> > [card] MCP Pre Tool
> >
> > Runs **before** the tool is called:
> >
> > * SQL injection prevention
> > * Parameter validation
> > * Permission checks (Cedar/OPA policies)
>
> > [card] MCP Post Tool
> >
> > Runs **after** the tool returns:
> >
> > * Code safety checks
> > * Secrets detection in outputs
> > * PII redaction from results
>
> ![Diagram showing the flow of MCP tool invocations through pre and post tool guardrails](https://mintcdn.com/truefoundry/Fc6z3zZxeYI2aatT/images/mcp_guardrails_flow.png?fit=max\&auto=format\&n=Fc6z3zZxeYI2aatT\&q=85\&s=af6c9d29abcfa2bb6bd4b73d39fbf129)
>
> MCP Tool Guardrail Flow
>
> The flow is straightforward:
>
> 1. **Pre Tool** guardrails all run before the tool is called. If any of them fail, the tool never executes.
> 2. The **tool runs** only after all pre-tool guardrails pass.
> 3. **Post Tool** guardrails check (and optionally rewrite) the tool's output.
> 4. The validated result goes back to the model.
>
> | Hook | Execution | What Happens on Failure |
> | - | - | - |
> | **MCP Pre Tool** | Sync (before tool invocation) | Tool doesn't run |
> | **MCP Post Tool** | Sync (after tool returns) | Result withheld from model |
>
> > [!NOTE]
> >
> > Guardrails run on **every tool call separately**. If an agent calls five tools in a row, each one gets its own guardrail checks.
>
> > [!TIP]
> >
> > MCP hooks matter most in agentic setups where the model decides which tools to call on its own. Use **Pre Tool** to stop bad actions before they happen (e.g., blocking a `DROP TABLE` query), and **Post Tool** to clean up results before the model sees them (e.g., redacting secrets from a database response).

## Latency Impact of Guardrails

Guardrails add processing time — but the AI Gateway is designed to keep that impact small.

> [tab] LLM Requests
>
> * **Input Validation runs in parallel** with the model request, so in the happy path, it adds no extra wait time before you see the first token.
> * **Input Mutation runs before** the model request, so its processing time is added directly.
> * When **Input Validation fails**, the model request gets cancelled immediately — you don't pay for a response you were going to throw away.
>
> **Execution Flow Examples**
>
> > [accordion] All Guardrails Pass
> >
> > Input validation runs in parallel with the model request. Output guardrails process the response before it's returned.
> >
> > ```mermaid theme={"dark"}
> > gantt
> >     title All Guardrails Pass
> >     dateFormat X
> >     axisFormat  
> >     section Input
> >     Input Mutation              :done, 0, 2
> >     section Parallel Execution
> >     Input Validation ✓          :done, 2, 5
> >     Model Request               :active, 2, 8
> >     section Output
> >     Output Mutation             :done, 8, 10
> >     Output Validation ✓         :done, 10, 12
> > ```
>
> > [accordion] Input Validation Failure
> >
> > Input validation fails while the model is running — the model request gets **cancelled immediately** to save costs.
> >
> > ```mermaid theme={"dark"}
> > gantt
> >     title Input Validation Failure — Model Request Cancelled
> >     dateFormat X
> >     axisFormat  
> >     section Input
> >     Input Mutation              :done, 0, 2
> >     section Parallel Execution
> >     Input Validation ✗          :crit, 2, 5
> >     Model Request (cancelled)   :crit, 2, 5
> > ```
>
> > [accordion] Output Validation Failure
> >
> > The model finishes, but output validation fails — the response is rejected. Model costs are already incurred at this point.
> >
> > ```mermaid theme={"dark"}
> > gantt
> >     title Output Validation Failure — Response Rejected
> >     dateFormat X
> >     axisFormat  
> >     section Input
> >     Input Mutation              :done, 0, 2
> >     section Parallel Execution
> >     Input Validation ✓          :done, 2, 5
> >     Model Request               :active, 2, 8
> >     section Output
> >     Output Mutation             :done, 8, 10
> >     Output Validation ✗         :crit, 10, 12
> > ```

> [tab] MCP Tool Invocations
>
> * **All MCP guardrails run synchronously** — pre-tool guardrails block before the tool executes, and post-tool guardrails block after.
> * **Pre-tool guardrail latency adds directly** to the tool call time since the tool cannot start until all pre-tool checks pass.
> * When a **pre-tool guardrail fails**, the tool never executes — you avoid the cost and side effects of a bad tool call entirely.

You can track the latency impact of each guardrail in **AI Gateway → Monitor → Request Traces**. Each guardrail span shows its execution time, result, scope, and which entity it was applied on.

![Request Traces view showing a selected Guardrail span with latency, result, scope, input, and output details](https://mintcdn.com/truefoundry/r1of9DsUNzanZ7Rp/images/trace-guardrail.png?fit=max\&auto=format\&n=r1of9DsUNzanZ7Rp\&q=85\&s=a0b4bd00c9da0eb3ca2ab1263414fd18)

Guardrail latency visible in Request Traces

## How to Apply Guardrails

Gateway policies attach guardrails to matching traffic. You can also add more guardrails on a single request with a header. The header is additive: those extra guardrails run together with the policies, and the policies still run.

> [accordion] Add extra guardrails on one request
>
> Pass the `X-TFY-GUARDRAILS` header to add LLM input or output guardrails for that request. Handy for testing, or when one request needs an extra check. Matching gateway policies still run. MCP tool hooks come from the policy, not this header.
>
> ```json theme={"dark"}
> {
>   "llm_input_guardrails": ["my-group/pii-redaction"],
>   "llm_output_guardrails": ["my-group/secrets-detection"]
> }
> ```

> [accordion] Gateway-Level via Policies
>
> First register your guardrails in **AI Gateway → Guardrails**, then set up guardrail policies in **AI Gateway → Policies → Guardrails** to apply them automatically based on who's making the request, which model they're calling, or which MCP tool is being used. This is the way to go for org-wide enforcement.
>
> For a step-by-step walkthrough, see the [Getting Started](https://www.truefoundry.com/docs/ai-gateway/guardrails-getting-started.md) guide. For the full policy guide, see [Configure Guardrail Policies](https://www.truefoundry.com/docs/ai-gateway/guardrails-configuration.md).

## Supported Guardrails

The AI Gateway ships with built-in guardrails and integrates with a range of external providers — all managed through a single interface.

### TrueFoundry Built-in Guardrails

Ready to use out of the box — no external credentials needed.

These integrations run on **TrueFoundry-managed** infrastructure: you configure them in the AI Gateway; TrueFoundry operates the guardrail execution path (no third-party API keys required for the built-ins listed here). If you need a **specific vendor** (for example OpenAI Moderations, Azure Content Safety, Bedrock Guardrails, or Google Model Armor), use an integration from [External Providers](#external-providers) or [Custom Guardrails](https://www.truefoundry.com/docs/ai-gateway/custom-guardrails.md).

Among TrueFoundry guardrails, these guardrails use the following services under the hood:

| Guardrail | Underlying Service | What It Does |
| - | - | - |
| [Content Moderation](https://www.truefoundry.com/docs/ai-gateway/tfy-content-moderation.md) | OpenAI Moderation | Detects harmful content across four categories: Hate (including harassment), Self-Harm, Sexual, and Violence (including illicit activity). Each category has a configurable score threshold. |
| [PII / PHI Detection](https://www.truefoundry.com/docs/ai-gateway/tfy-pii.md) | Azure AI Language — PII Detection | Detects and optionally redacts personally identifiable information (PII) and protected health information (PHI) such as names, addresses, SSNs, and medical record numbers. |
| [Prompt Injection](https://www.truefoundry.com/docs/ai-gateway/tfy-prompt-injection.md) | Azure AI Content Safety — Prompt Shield | Detects jailbreak attempts and prompt injection attacks designed to manipulate LLM behavior. |

> [!WARNING]
>
> These three guardrails are **only available on TrueFoundry SaaS**. They are not available on self-hosted deployments. In case of non-SaaS (self-hosted) deployments, you can use the following alternatives:
>
> | Instead of | Consider Using |
> | - | - |
> | Content Moderation | [OpenAI Moderations](https://www.truefoundry.com/docs/ai-gateway/openai-moderations.md), [AWS Bedrock Guardrails](https://www.truefoundry.com/docs/ai-gateway/bedrock-guardrails.md), [Enkrypt AI](https://www.truefoundry.com/docs/ai-gateway/enkrypt-ai.md) |
> | PII / PHI Detection | [Azure PII](https://www.truefoundry.com/docs/ai-gateway/azure-pii.md) (bring your own key), [CrowdStrike](https://www.truefoundry.com/docs/ai-gateway/crowstrike.md) |
> | Prompt Injection | [Azure Prompt Shield](https://www.truefoundry.com/docs/ai-gateway/azure-prompt-shield.md) (bring your own key), [Palo Alto Prisma AIRS](https://www.truefoundry.com/docs/ai-gateway/palo-alto-airs.md) |

> [card] [Secrets Detection](https://www.truefoundry.com/docs/ai-gateway/secrets-detection.md)
>
> Catches and redacts credentials like AWS keys, API keys, JWT tokens, and private keys.

> [card] [Code Safety Linter](https://www.truefoundry.com/docs/ai-gateway/code-safety-linter.md)
>
> Flags unsafe code patterns — eval, exec, os.system, subprocess calls, dangerous shell commands.

> [card] [SQL Sanitizer](https://www.truefoundry.com/docs/ai-gateway/sql-sanitizer.md)
>
> Catches risky SQL: DROP, TRUNCATE, DELETE/UPDATE without WHERE, string interpolation.

> [card] [Regex Pattern Matching](https://www.truefoundry.com/docs/ai-gateway/regex-pattern-matching.md)
>
> Matches and redacts sensitive patterns (PII, payment cards, credentials) using built-in or custom regex.

> [card] [Prompt Injection](https://www.truefoundry.com/docs/ai-gateway/tfy-prompt-injection.md)
>
> Detects prompt injection attacks and jailbreak attempts using model-based analysis.

> [card] [PII Detection](https://www.truefoundry.com/docs/ai-gateway/tfy-pii.md)
>
> Finds and redacts personally identifiable information with configurable entity categories.

> [card] [Content Moderation](https://www.truefoundry.com/docs/ai-gateway/tfy-content-moderation.md)
>
> Blocks harmful content across hate, self-harm, sexual, and violence categories with adjustable thresholds.

> [card] [Cedar Guardrails](https://www.truefoundry.com/docs/ai-gateway/cedar-guardrails.md)
>
> Fine-grained access control for MCP tools using Cedar policy language with default-deny security.

> [card] [OPA Guardrails](https://www.truefoundry.com/docs/ai-gateway/opa-guardrails.md)
>
> Fine-grained access control with full policy lifecycle management using Open Policy Agent.

### External Providers

We also integrate with third-party guardrail providers. Don't see yours? Reach out — we're happy to add it.

> [card] [Akto](https://www.truefoundry.com/docs/ai-gateway/akto.md)
>
> LLM security, prompt injection detection, and policy violation monitoring with native streaming support.

> [card] [AWS Bedrock Guardrail](https://www.truefoundry.com/docs/ai-gateway/bedrock-guardrails.md)
>
> AWS Bedrock's guardrail capabilities for AI models.

> [card] [Azure Content Safety](https://www.truefoundry.com/docs/ai-gateway/azure-content-safety.md)
>
> Azure Content Safety for detecting harmful or inappropriate content.

> [card] [Azure PII](https://www.truefoundry.com/docs/ai-gateway/azure-pii.md)
>
> Azure's PII detection service for identifying and redacting personal data.

> [card] [Azure Prompt Shield](https://www.truefoundry.com/docs/ai-gateway/azure-prompt-shield.md)
>
> Azure Prompt Shield for blocking prompt injection and jailbreak attempts.

> [card] [CrowdStrike](https://www.truefoundry.com/docs/ai-gateway/crowdstrike.md)
>
> API-based security — content moderation, prompt injection detection, toxicity analysis. (Formerly Pangea, acquired by CrowdStrike in 2025.)

> [card] [Cisco AI Defense](https://www.truefoundry.com/docs/ai-gateway/cisco-ai-defense.md)
>
> Inspect LLM prompts and responses for security, safety, privacy, and relevance violations.

> [card] [Enkrypt AI](https://www.truefoundry.com/docs/ai-gateway/enkrypt-ai.md)
>
> Advanced moderation and compliance — toxicity, bias, and sensitive data detection.

> [card] [F5 AI Security](https://www.truefoundry.com/docs/ai-gateway/f5-calypsoai.md)
>
> Scan LLM prompts and responses for prompt injection, jailbreak, PII, and policy violations.

> [card] [Fiddler](https://www.truefoundry.com/docs/ai-gateway/fiddler.md)
>
> Fiddler-Safety and Fiddler-Response-Faithfulness guardrails.

> [card] [Google Model Armor](https://www.truefoundry.com/docs/ai-gateway/google-model-armor.md)
>
> Google Cloud Model Armor for prompt injection, harmful content, PII, and malicious URI detection.

> [card] [GraySwan Cygnal](https://www.truefoundry.com/docs/ai-gateway/grayswan-cygnal.md)
>
> Policy violation detection and content safety monitoring powered by GraySwan Cygnal.

> [card] [HiddenLayer](https://www.truefoundry.com/docs/ai-gateway/hiddenlayer.md)
>
> HiddenLayer AIDR for prompt injection, PII, and policy violations with project-policy-driven redaction.

> [card] [Fireworks](https://www.truefoundry.com/docs/ai-gateway/fireworks.md)
>
> Fireworks-hosted safety models (gpt-oss-safeguard with your policy, or Llama Guard) classify prompts and responses.

> [card] [OpenAI Moderations](https://www.truefoundry.com/docs/ai-gateway/openai-moderations.md)
>
> OpenAI's moderation API for detecting violence, hate speech, harassment, and other policy violations.

> [card] [Palo Alto Prisma AIRS](https://www.truefoundry.com/docs/ai-gateway/palo-alto-airs.md)
>
> Palo Alto AI Risk for content safety and threat detection.

> [card] [Patronus AI](https://www.truefoundry.com/docs/ai-gateway/patronus.md)
>
> Hallucination detection, prompt injection, PII leakage, toxicity, and bias evaluators.

> [card] [Pillar Security](https://www.truefoundry.com/docs/ai-gateway/pillar-security.md)
>
> Pillar Security guardrails for LLM application protection.

> [card] [Straiker](https://www.truefoundry.com/docs/ai-gateway/straiker.md)
>
> Checks LLM prompts and responses against the agent controls you set in the Straiker console.

> [card] [TrojAI](https://www.truefoundry.com/docs/ai-gateway/trojai.md)
>
> LLM security testing and runtime guardrails from TrojAI.

> [card] [Zscaler AI Guard](https://www.truefoundry.com/docs/ai-gateway/zscaler-ai-guard.md)
>
> Blocks or masks prompts, model responses, and MCP tool calls.

### Custom guardrail wrappers

> [accordion] External Providers
>
> We also integrate with third-party guardrail providers. Don't see yours? Reach out — we're happy to add it.
>
> > [card] [OpenAI Moderations](https://www.truefoundry.com/docs/ai-gateway/openai-moderations.md)
>
> > [card] [AWS Bedrock Guardrail](https://www.truefoundry.com/docs/ai-gateway/bedrock-guardrails.md)
>
> > [card] [Azure PII](https://www.truefoundry.com/docs/ai-gateway/azure-pii.md)
>
> > [card] [Azure Content Safety](https://www.truefoundry.com/docs/ai-gateway/azure-content-safety.md)
>
> > [card] [Azure Prompt Shield](https://www.truefoundry.com/docs/ai-gateway/azure-prompt-shield.md)
>
> > [card] [Enkrypt AI](https://www.truefoundry.com/docs/ai-gateway/enkrypt-ai.md)
>
> > [card] [Palo Alto Prisma AIRS](https://www.truefoundry.com/docs/ai-gateway/palo-alto-airs.md)
>
> > [card] [Fiddler](https://www.truefoundry.com/docs/ai-gateway/fiddler.md)
>
> > [card] [CrowdStrike](https://www.truefoundry.com/docs/ai-gateway/crowdstrike.md)
>
> > [card] [Cisco AI Defense](https://www.truefoundry.com/docs/ai-gateway/cisco-ai-defense.md)
>
> > [card] [Cato Networks](https://www.truefoundry.com/docs/ai-gateway/cato-networks.md)
>
> > [card] [Zscaler AI Guard](https://www.truefoundry.com/docs/ai-gateway/zscaler-ai-guard.md)
>
> > [card] [F5 AI Security](https://www.truefoundry.com/docs/ai-gateway/f5-calypsoai.md)
>
> > [card] [Patronus AI](https://www.truefoundry.com/docs/ai-gateway/patronus.md)
>
> > [card] [Google Model Armor](https://www.truefoundry.com/docs/ai-gateway/google-model-armor.md)
>
> > [card] [Gray Swan Cygnal](https://www.truefoundry.com/docs/ai-gateway/grayswan-cygnal.md)
>
> > [card] [HiddenLayer](https://www.truefoundry.com/docs/ai-gateway/hiddenlayer.md)
>
> > [card] [Fireworks](https://www.truefoundry.com/docs/ai-gateway/fireworks.md)
>
> > [card] [Akto](https://www.truefoundry.com/docs/ai-gateway/akto.md)
>
> > [card] [TrojAI](https://www.truefoundry.com/docs/ai-gateway/trojai.md)
>
> > [card] [Noma Security](https://www.truefoundry.com/docs/ai-gateway/noma-security.md)
>
> > [card] [Pillar Security](https://www.truefoundry.com/docs/ai-gateway/pillar-security.md)
>
> > [card] [Straiker](https://www.truefoundry.com/docs/ai-gateway/straiker.md)

> [card] [Arthur AI](https://www.truefoundry.com/docs/ai-gateway/arthur-ai.md)
>
> Arthur GenAI Engine stateless validation for prompt injection, toxicity, and hallucination—validate-only SaaS.

> [card] [DeepKeep](https://www.truefoundry.com/docs/ai-gateway/deepkeep.md)
>
> DeepKeep AI Firewall for PII replace, secret-key blocking, prompt-injection defense, and toxic-language filtering.

> [card] [Guardrails AI](https://www.truefoundry.com/docs/ai-gateway/guardrails-ai.md)
>
> Hub validators for PII, secrets, toxicity, and profanity—local heuristics in the wrapper pod.

> [card] [Lasso Security](https://www.truefoundry.com/docs/ai-gateway/lasso-security.md)
>
> Lasso API v3 `classify` and `classifix` for validate and mutate (PII masking)—policy in the Lasso console.

> [card] [NVIDIA NeMo](https://www.truefoundry.com/docs/ai-gateway/nvidia-nemo.md)
>
> NeMo `self_check_input` / `self_check_output` for jailbreak and output safety (judge LLM via your gateway).

> [card] [Verra](https://www.truefoundry.com/docs/ai-gateway/verra.md)
>
> Managed AI governance for regulated industries—validate and mutate with PII/secrets redaction and compliant audit trail.

### Bring Your Own Guardrail / Plugin

If the built-in and provider integrations don't cover your use case, you can write your own. Build a custom guardrail with Guardrails.AI, a plain Python function, or any framework you prefer, and plug it into the AI Gateway.

> [card] [Custom Guardrails](https://www.truefoundry.com/docs/ai-gateway/custom-guardrails.md)
>
> Build and integrate your own guardrail using the template repo, or start from the [reference integrations](https://www.truefoundry.com/docs/ai-gateway/custom-guardrails.md#reference-integrations-deployable-wrappers) above.

## FAQ

> [accordion] How do I control which messages guardrails evaluate?
>
> By default, guardrails look at **all messages** in the conversation. Use the `X-TFY-GUARDRAILS-SCOPE` header to narrow that:
>
> * **`all`** (default): Checks the full conversation history
> * **`last`**: Checks only the most recent message
> * **positive integer** (e.g. `10`): Checks the last N messages
>
> Use a last-N window when a single request can append several new turns at once (for example tool results followed by a user message in an agent loop). Checking only `last` would miss the new-but-not-final messages in that request. Pure one-message-per-request chat can use `last`.
>
> MCP tool pre/post guardrails always scan the full tool arguments or result — scope does not apply there. Oversized text is still split into character-limited chunks for providers with per-request limits.
>
> ```bash theme={"dark"}
> curl -X POST "{GATEWAY_BASE_URL}/chat/completions" \
>   -H "Authorization: Bearer YOUR_API_KEY" \
>   -H "Content-Type: application/json" \
>   -H 'X-TFY-GUARDRAILS: {"llm_input_guardrails":["my-group/pii-redaction"]}' \
>   -H "X-TFY-GUARDRAILS-SCOPE: 10" \
>   -d '{
>     "model": "openai/gpt-4o",
>     "messages": [
>       {"role": "system", "content": "You are a helpful assistant."},
>       {"role": "user", "content": "Hello, how can you help me today?"}
>     ]
>   }'
> ```

> [accordion] What happens if a guardrail service is down?
>
> Depends on the enforcement strategy you picked:
>
> * **Enforce**: Request gets blocked.
> * **Enforce But Ignore On Error**: Request goes through anyway. This is the safest default — you stay protected when guardrails work, but a provider outage won't break your app.
> * **Audit**: Request always goes through.

> [accordion] How can I monitor guardrail execution?
>
> Go to **AI Gateway → Monitor → Request Traces**. You'll see:
>
> * Which guardrails ran on each hook
> * Whether they passed or failed, and how long they took
> * What they found (secrets, SQL issues, unsafe patterns, etc.)
> * What mutations were applied
>
> Traces are logged for both successful and blocked requests.

> [accordion] Do output guardrails work with streaming responses?
>
> **No.** Output guardrails are **skipped** when the response is streamed (`"stream": true`). Output guardrails need the complete response text to evaluate, but streaming sends tokens to the client as they are generated — so there is no full response to check before delivery begins.
>
> **What you can do:**
>
> * **Set `"stream": false`** if you need output guardrails to run. This ensures the AI Gateway receives the full response, evaluates it against your output guardrails, and only then returns it to the client.
> * **Handle checks client-side** — assemble the full streamed response on the client and run your own output validation after the stream completes.
> * **Use input-only guardrails with streaming** — input guardrails always run regardless of streaming mode, so you still get protection on the prompt side.
>
> ```python theme={"dark"}
> # Output guardrails WILL run:
> response = client.chat.completions.create(
>     model="my-model",
>     messages=[{"role": "user", "content": "Hello"}],
>     stream=False  # output guardrails are applied
> )
>
> # Output guardrails will NOT run:
> response = client.chat.completions.create(
>     model="my-model",
>     messages=[{"role": "user", "content": "Hello"}],
>     stream=True  # output guardrails are skipped
> )
> ```

> [accordion] Are system prompts evaluated by guardrails?
>
> **No.** System prompts are excluded from guardrails by default — the AI Gateway strips them before sending content to any guardrail, so they are never inspected, blocked, or redacted.
>
> The one exception is [CrowdStrike AIDR](https://www.truefoundry.com/docs/ai-gateway/crowdstrike.md), which sees the system prompt for analysis but never modifies it.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.