> ## Documentation Index
> Fetch the complete documentation index at: https://www.truefoundry.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Guardrails Best Practices

> Which guardrail to attach to which hook — LLM Input, LLM Output, MCP Tool Pre-Invoke, or MCP Tool Post-Invoke — and why.

A guardrail only protects you if it runs where the risk enters. Most guardrails *can* be attached to any of the four hooks, but each one catches the most, at the least latency and cost, on one or two of them. This page tells you which hook to use for each type of guardrail.

The AI Gateway has four hooks. You pick them under **Apply on Hooks** in a [guardrail policy](https://www.truefoundry.com/docs/ai-gateway/guardrails-configuration.md#apply-on-hooks):

| Hook | Runs on | YAML field |
| - | - | - |
| **LLM Input** | The prompt, before it reaches the model | `llm_input_guardrails` |
| **LLM Output** | The model's response, before it reaches the client | `llm_output_guardrails` |
| **MCP Tool Pre-Invoke** | The tool's arguments, before the MCP tool runs | `mcp_tool_pre_invoke_guardrails` |
| **MCP Tool Post-Invoke** | The tool's result, before it goes back to the model | `mcp_tool_post_invoke_guardrails` |

## Which Guardrail Goes on Which Hook

| Guardrail type | LLM Input | LLM Output | MCP Tool Pre-Invoke | MCP Tool Post-Invoke | Operation |
| - | :-: | :-: | :-: | :-: | - |
| [Prompt Injection](https://www.truefoundry.com/docs/ai-gateway/tfy-prompt-injection.md) | ✅ | | | ✅ | `validate` |
| [Content Moderation](https://www.truefoundry.com/docs/ai-gateway/tfy-content-moderation.md) | ✅ | | | | `validate` |
| [PII / PHI Detection](https://www.truefoundry.com/docs/ai-gateway/tfy-pii.md) | ✅ | | ✅ | | `mutate` |
| [Secrets Detection](https://www.truefoundry.com/docs/ai-gateway/secrets-detection.md) | ✅ | | | ✅ | `mutate` |
| [Regex Pattern Matching](https://www.truefoundry.com/docs/ai-gateway/regex-pattern-matching.md) | ✅ | | ✅ | | `mutate` |
| [SQL Sanitizer](https://www.truefoundry.com/docs/ai-gateway/sql-sanitizer.md) | | | ✅ | | `validate` |
| [Code Safety Linter](https://www.truefoundry.com/docs/ai-gateway/code-safety-linter.md) | | ✅ | ✅ | | `validate` |
| [Cedar](https://www.truefoundry.com/docs/ai-gateway/cedar-guardrails.md) / [OPA](https://www.truefoundry.com/docs/ai-gateway/opa-guardrails.md) policies | | | ✅ | | `validate` |
| [Metadata Validation](https://www.truefoundry.com/docs/ai-gateway/metadata-validation.md) | ✅ | | | | `validate` |
| Hallucination / response quality (for example [Fiddler](https://www.truefoundry.com/docs/ai-gateway/fiddler.md), [Patronus AI](https://www.truefoundry.com/docs/ai-gateway/patronus.md)) | | ✅ | | | `validate` |

The same placement applies when you use an [external provider](https://www.truefoundry.com/docs/ai-gateway/guardrails-overview.md#external-providers) for that type of check — for example, [Azure Prompt Shield](https://www.truefoundry.com/docs/ai-gateway/azure-prompt-shield.md) goes where Prompt Injection goes, and [OpenAI Moderations](https://www.truefoundry.com/docs/ai-gateway/openai-moderations.md) goes where Content Moderation goes.

```mermaid theme={"dark"}
flowchart LR
    Client([Client]) --> LI["LLM Input<br/>Prompt Injection · Content Moderation<br/>PII/PHI · Secrets · Regex · Metadata"]
    LI --> LLM[(LLM)]
    LLM --> LO["LLM Output<br/>Code Safety · Hallucination"]
    LO --> Client
    LLM -. tool call .-> Pre["MCP Tool Pre-Invoke<br/>PII/PHI · Regex · SQL Sanitizer<br/>Code Safety · Cedar / OPA"]
    Pre --> Tool[(MCP tool)]
    Tool --> Post["MCP Tool Post-Invoke<br/>Prompt Injection · Secrets"]
    Post -. tool result .-> LLM
```

## Why Each Guardrail Goes Where It Does

### Prompt Injection — LLM Input and MCP Tool Post-Invoke

Prompt injection reaches the model in two ways, and each needs its own hook:

* **Direct injection and jailbreaks** come from the user. On **LLM Input**, the guardrail blocks them before the model sees them. Because it runs in `validate` mode, it runs in parallel with the model request and adds no time to first token when it passes. When it fails, the gateway cancels the model request.
* **Indirect injection** comes from content the agent pulls in: a web page, an email, a file, a database row. That content enters through tool results. On **MCP Tool Post-Invoke**, the guardrail checks every tool result before the model reads it.

You need both. On LLM Input, the TrueFoundry Prompt Injection guardrail scans the **last two user messages** only — it does not scan tool results in the conversation. Tool results are covered by the MCP Tool Post-Invoke hook.

Don't add Prompt Injection on **LLM Output**. The model's own response is not an injection vector, and the Prompt Shield service behind the guardrail does not support that hook.

### Content Moderation — LLM Input

Content Moderation stops harmful requests (hate, self-harm, sexual, violence) before you pay for a response. On **LLM Input**, it runs in parallel with the model request, so it adds no latency when it passes, and a failure cancels the model call.

Leave it off the other hooks:

* **LLM Output**: output guardrails don't run on streamed responses (`"stream": true`), which covers most chat and agent traffic. When they do run, the model has already been paid for.
* **MCP hooks**: tool arguments and results are machine-generated data, not user-written content. Moderating every tool call adds latency to each hop of an agent loop without catching much.

### PII / PHI Detection — LLM Input and MCP Tool Pre-Invoke

Put PII/PHI redaction at the points where data **leaves** your boundary:

* **LLM Input** redacts PII/PHI from the prompt before it reaches the model provider. The model never sees the data, so it can't repeat it in its response either.
* **MCP Tool Pre-Invoke** redacts PII/PHI from tool arguments before they reach the MCP server — for example, a search query or a ticket body that the agent sends to a third-party tool.

Use the `mutate` operation so the request continues with the sensitive values masked, instead of being blocked. Mutate guardrails run **before** the model request, so their latency adds to time to first token — keep the entity categories to the ones you need.

Don't add it on **MCP Tool Post-Invoke** by default. Tool results often *need* the identifiers they return (a CRM lookup returns the customer's email so the agent can act on it), and redacting them breaks the workflow. Results are also large, so scanning them adds latency to every tool call. Anything in a tool result that the model later repeats is caught on the next LLM Input.

### Secrets Detection — LLM Input and MCP Tool Post-Invoke

* **LLM Input** catches credentials that users paste into a prompt (a stack trace with an API key, a `.env` file) before they are sent to the model provider.
* **MCP Tool Post-Invoke** catches credentials in tool results — file reads, environment dumps, config fetches, database rows — before the model reads them and can repeat them.

Use `mutate` to redact the secret and keep the request going. Add **LLM Output** as well only for non-streaming apps where the model could generate a credential from its own context.

### Regex Pattern Matching — LLM Input and MCP Tool Pre-Invoke

Regex covers fixed-format identifiers that a model-based PII detector doesn't know about — employee IDs, internal account numbers, project codenames. Place it with PII/PHI Detection, on **LLM Input** and **MCP Tool Pre-Invoke**, in `mutate` mode. It runs locally in the gateway, so it adds very little latency.

### SQL Sanitizer — MCP Tool Pre-Invoke

SQL is dangerous when it runs, so check it **before** a database tool executes it. On **MCP Tool Pre-Invoke**, a `DROP TABLE` or a `DELETE` without `WHERE` is blocked and the tool never runs. Scope the policy to your database MCP servers or tools with the `mcpServers` / `mcpTools` [targets](https://www.truefoundry.com/docs/ai-gateway/guardrails-configuration.md#the-when-block).

### Code Safety Linter — MCP Tool Pre-Invoke and LLM Output

* **MCP Tool Pre-Invoke** on code-execution and shell tools blocks `rm -rf`, `eval`, `os.system`, and similar patterns in the tool's arguments **before** the code runs. This is the hook that prevents damage. Scope it to those tools only.
* **LLM Output** flags unsafe code in model responses when the code is shown to a user rather than run by a tool. It only applies to non-streaming requests.

### Cedar / OPA — MCP Tool Pre-Invoke

Access policies decide **whether a tool call is allowed** — who can call which tool, with which arguments. That decision has to happen before the tool runs, so attach them to **MCP Tool Pre-Invoke**. [OPA](https://www.truefoundry.com/docs/ai-gateway/opa-guardrails.md) can also run on LLM Input when you need request-level rules (for example, which teams may call which model with which parameters).

### Metadata Validation — LLM Input

Metadata Validation checks the `X-TFY-METADATA` header of the request, so it only does anything on **LLM Input**. On every other hook it passes without checking.

### Hallucination and response quality — LLM Output

Checks such as faithfulness or groundedness need the complete answer, so they belong on **LLM Output**. They don't run on streamed responses — set `"stream": false` for the traffic you want checked.

## Recommended Baseline Policy

A starting point that applies the placement above to all models and MCP servers. Replace the selectors with the guardrails you registered in **AI Gateway → Guardrails**:

```yaml theme={"dark"}
type: gateway-guardrails-config
name: baseline-guardrails
when:
  target:
    operator: or
    conditions:
      model:
        values:
          - "*"
        condition: in
      mcpServers:
        values:
          - "*"
        condition: in
llm_input_guardrails:
  - tfy-guardrails/pii-detection
  - tfy-guardrails/prompt-injection
  - tfy-guardrails/content-moderation
  - tfy-guardrails/secrets-detection
llm_output_guardrails: []
mcp_tool_pre_invoke_guardrails:
  - tfy-guardrails/pii-detection
mcp_tool_post_invoke_guardrails:
  - tfy-guardrails/prompt-injection
  - tfy-guardrails/secrets-detection
```

Then add narrower policies for specific tools — for example, SQL Sanitizer on MCP Tool Pre-Invoke for your database MCP server only, and Code Safety Linter on MCP Tool Pre-Invoke for your code-execution tools only. When several policies match a request, their guardrails are [merged per hook](https://www.truefoundry.com/docs/ai-gateway/guardrails-configuration.md#how-policies-are-evaluated), so the baseline still runs.

## General Recommendations

> [accordion] Prefer earlier hooks
>
> A check on **LLM Input** or **MCP Tool Pre-Invoke** stops the problem before the model call or tool call happens. A check on **LLM Output** runs after the model has already been paid for, and a check on **MCP Tool Post-Invoke** runs after the tool's side effects have already happened. If a risk can be caught before the call, catch it there.

> [accordion] Don't rely on LLM Output for streaming or agent traffic
>
> LLM Output guardrails are skipped when `"stream": true`. Chat UIs and coding agents almost always stream, so for that traffic put your checks on LLM Input and the MCP hooks. Use LLM Output only for non-streaming APIs.

> [accordion] Use validate where you can, mutate where you must
>
> On LLM Input, `validate` guardrails run in parallel with the model request; `mutate` guardrails run before it and add their latency to time to first token. Use `mutate` only for redaction (PII/PHI, Secrets, Regex) and `validate` for everything that only needs to block.

> [accordion] Scope MCP guardrails to the tools that need them
>
> MCP guardrails run on **every** tool call, and an agent may make dozens of calls per task. Attach SQL Sanitizer and Code Safety Linter only to the MCP servers or tools that run SQL or code, using `mcpServers` / `mcpTools` targets.

> [accordion] Tools that don't go through the gateway's MCP hooks
>
> MCP hooks run only on tool calls that go through the AI Gateway (MCP servers registered on the gateway). Tools that the client runs itself — for example, a coding agent's local shell or file tools — don't trigger them; their results come back to the gateway as messages in the next LLM request. To check those results for indirect prompt injection, use a prompt-injection guardrail on LLM Input that scans the whole conversation, such as [Azure Prompt Shield](https://www.truefoundry.com/docs/ai-gateway/azure-prompt-shield.md) with your own credentials, together with the [`X-TFY-GUARDRAILS-SCOPE`](https://www.truefoundry.com/docs/ai-gateway/guardrails-overview#faq) header.

> [accordion] Roll out in Audit first
>
> Start every new guardrail with the **Audit** enforcing strategy and check **AI Gateway → Monitor → Request Traces** for what it would have blocked. Move to **Enforce But Ignore On Error** once false positives are acceptable, and to **Enforce** only for checks that must never be skipped. See [Enforcement Strategy](https://www.truefoundry.com/docs/ai-gateway/guardrails-overview.md#enforcement-strategy).

> [accordion] Self-hosted gateways
>
> The TrueFoundry-managed Prompt Injection, Content Moderation, and PII/PHI Detection guardrails are available only when TrueFoundry hosts the gateway. On self-hosted and hybrid deployments, use the [alternatives listed here](https://www.truefoundry.com/docs/ai-gateway/truefoundry-guardrails.md#where-to-add-these-guardrails) on the same hooks.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.