Skip to main content
A guardrail only protects you if it runs where the risk enters. Most guardrails can be attached to any of the four hooks, but each one catches the most, at the least latency and cost, on one or two of them. This page tells you which hook to use for each type of guardrail. The AI Gateway has four hooks. You pick them under Apply on Hooks in a guardrail policy:

Which Guardrail Goes on Which Hook

The same placement applies when you use an external provider for that type of check — for example, Azure Prompt Shield goes where Prompt Injection goes, and OpenAI Moderations goes where Content Moderation goes.

Why Each Guardrail Goes Where It Does

Prompt Injection — LLM Input and MCP Tool Post-Invoke

Prompt injection reaches the model in two ways, and each needs its own hook:
  • Direct injection and jailbreaks come from the user. On LLM Input, the guardrail blocks them before the model sees them. Because it runs in validate mode, it runs in parallel with the model request and adds no time to first token when it passes. When it fails, the gateway cancels the model request.
  • Indirect injection comes from content the agent pulls in: a web page, an email, a file, a database row. That content enters through tool results. On MCP Tool Post-Invoke, the guardrail checks every tool result before the model reads it.
You need both. On LLM Input, the TrueFoundry Prompt Injection guardrail scans the last two user messages only — it does not scan tool results in the conversation. Tool results are covered by the MCP Tool Post-Invoke hook. Don’t add Prompt Injection on LLM Output. The model’s own response is not an injection vector, and the Prompt Shield service behind the guardrail does not support that hook.

Content Moderation — LLM Input

Content Moderation stops harmful requests (hate, self-harm, sexual, violence) before you pay for a response. On LLM Input, it runs in parallel with the model request, so it adds no latency when it passes, and a failure cancels the model call. Leave it off the other hooks:
  • LLM Output: output guardrails don’t run on streamed responses ("stream": true), which covers most chat and agent traffic. When they do run, the model has already been paid for.
  • MCP hooks: tool arguments and results are machine-generated data, not user-written content. Moderating every tool call adds latency to each hop of an agent loop without catching much.

PII / PHI Detection — LLM Input and MCP Tool Pre-Invoke

Put PII/PHI redaction at the points where data leaves your boundary:
  • LLM Input redacts PII/PHI from the prompt before it reaches the model provider. The model never sees the data, so it can’t repeat it in its response either.
  • MCP Tool Pre-Invoke redacts PII/PHI from tool arguments before they reach the MCP server — for example, a search query or a ticket body that the agent sends to a third-party tool.
Use the mutate operation so the request continues with the sensitive values masked, instead of being blocked. Mutate guardrails run before the model request, so their latency adds to time to first token — keep the entity categories to the ones you need. Don’t add it on MCP Tool Post-Invoke by default. Tool results often need the identifiers they return (a CRM lookup returns the customer’s email so the agent can act on it), and redacting them breaks the workflow. Results are also large, so scanning them adds latency to every tool call. Anything in a tool result that the model later repeats is caught on the next LLM Input.

Secrets Detection — LLM Input and MCP Tool Post-Invoke

  • LLM Input catches credentials that users paste into a prompt (a stack trace with an API key, a .env file) before they are sent to the model provider.
  • MCP Tool Post-Invoke catches credentials in tool results — file reads, environment dumps, config fetches, database rows — before the model reads them and can repeat them.
Use mutate to redact the secret and keep the request going. Add LLM Output as well only for non-streaming apps where the model could generate a credential from its own context.

Regex Pattern Matching — LLM Input and MCP Tool Pre-Invoke

Regex covers fixed-format identifiers that a model-based PII detector doesn’t know about — employee IDs, internal account numbers, project codenames. Place it with PII/PHI Detection, on LLM Input and MCP Tool Pre-Invoke, in mutate mode. It runs locally in the gateway, so it adds very little latency.

SQL Sanitizer — MCP Tool Pre-Invoke

SQL is dangerous when it runs, so check it before a database tool executes it. On MCP Tool Pre-Invoke, a DROP TABLE or a DELETE without WHERE is blocked and the tool never runs. Scope the policy to your database MCP servers or tools with the mcpServers / mcpTools targets.

Code Safety Linter — MCP Tool Pre-Invoke and LLM Output

  • MCP Tool Pre-Invoke on code-execution and shell tools blocks rm -rf, eval, os.system, and similar patterns in the tool’s arguments before the code runs. This is the hook that prevents damage. Scope it to those tools only.
  • LLM Output flags unsafe code in model responses when the code is shown to a user rather than run by a tool. It only applies to non-streaming requests.

Cedar / OPA — MCP Tool Pre-Invoke

Access policies decide whether a tool call is allowed — who can call which tool, with which arguments. That decision has to happen before the tool runs, so attach them to MCP Tool Pre-Invoke. OPA can also run on LLM Input when you need request-level rules (for example, which teams may call which model with which parameters).

Metadata Validation — LLM Input

Metadata Validation checks the X-TFY-METADATA header of the request, so it only does anything on LLM Input. On every other hook it passes without checking.

Hallucination and response quality — LLM Output

Checks such as faithfulness or groundedness need the complete answer, so they belong on LLM Output. They don’t run on streamed responses — set "stream": false for the traffic you want checked. A starting point that applies the placement above to all models and MCP servers. Replace the selectors with the guardrails you registered in AI Gateway → Guardrails:
Then add narrower policies for specific tools — for example, SQL Sanitizer on MCP Tool Pre-Invoke for your database MCP server only, and Code Safety Linter on MCP Tool Pre-Invoke for your code-execution tools only. When several policies match a request, their guardrails are merged per hook, so the baseline still runs.

General Recommendations

A check on LLM Input or MCP Tool Pre-Invoke stops the problem before the model call or tool call happens. A check on LLM Output runs after the model has already been paid for, and a check on MCP Tool Post-Invoke runs after the tool’s side effects have already happened. If a risk can be caught before the call, catch it there.
LLM Output guardrails are skipped when "stream": true. Chat UIs and coding agents almost always stream, so for that traffic put your checks on LLM Input and the MCP hooks. Use LLM Output only for non-streaming APIs.
On LLM Input, validate guardrails run in parallel with the model request; mutate guardrails run before it and add their latency to time to first token. Use mutate only for redaction (PII/PHI, Secrets, Regex) and validate for everything that only needs to block.
MCP guardrails run on every tool call, and an agent may make dozens of calls per task. Attach SQL Sanitizer and Code Safety Linter only to the MCP servers or tools that run SQL or code, using mcpServers / mcpTools targets.
MCP hooks run only on tool calls that go through the AI Gateway (MCP servers registered on the gateway). Tools that the client runs itself — for example, a coding agent’s local shell or file tools — don’t trigger them; their results come back to the gateway as messages in the next LLM request. To check those results for indirect prompt injection, use a prompt-injection guardrail on LLM Input that scans the whole conversation, such as Azure Prompt Shield with your own credentials, together with the X-TFY-GUARDRAILS-SCOPE header.
Start every new guardrail with the Audit enforcing strategy and check AI Gateway → Monitor → Request Traces for what it would have blocked. Move to Enforce But Ignore On Error once false positives are acceptable, and to Enforce only for checks that must never be skipped. See Enforcement Strategy.
The TrueFoundry-managed Prompt Injection, Content Moderation, and PII/PHI Detection guardrails are available only when TrueFoundry hosts the gateway. On self-hosted and hybrid deployments, use the alternatives listed here on the same hooks.