Data Masking in the AI Gateway: What Actually Works

What data masking actually is

Data masking replaces a sensitive value with a substitute that is safe to expose, while keeping the surrounding data usable. It comes from the database world: give QA a copy of production where every card number is 4111-****-****-1234 and they can test realistic shapes without holding real PII. Point that at an LLM gateway and it changes — you are masking free text, on its way to a third party, inside a conversation that has to keep making sense.

These get used as synonyms constantly. They are not synonyms.

Reversible vs irreversible is the distinction that matters. Tokenisation, FPE and encryption round-trip — someone holds a vault or a key, a new asset to protect. Masking, redaction and anonymisation are one-way: simpler, safer, and useless the moment something downstream needs the real value.

Pseudonymisation is not anonymisation. Under GDPR, pseudonymised data is still personal data and still in scope; only genuinely anonymised data falls outside the regulation (Recital 26). Replacing every name with a stable User-8812 is pseudonymisation, and calling it the latter will not survive review.

A third axis is easy to miss: consistency is separate from reversibility. A masker can be deterministic — the same input always yields the same placeholder — without being reversible, and determinism is what lets a model reason about “the same person” across a conversation.

What breaks when you mask LLM traffic

This is the part most write-ups skip, and it decides whether your design survives.

Model output quality degrades, unevenly. Turn Call our office at 312-555-1234 into Call our office at ************ and the model no longer knows that was a phone number — asterisks carry no type information. Labelled placeholders like [PHONE_NUMBER] keep the type, at the cost of telling anyone who sees the prompt what was there. Summarisation barely notices; extraction fails loudly; correlating two entities fails quietly, which is worse.

Tool calls need the real value. The sharpest failure. An agent that masks customer@acme.com on the way in cannot then call lookup_account(email=...). Masking the model path and leaving the tool path open creates the opposite hole: the model never saw the address, but the tool arguments carry it to a third-party MCP server. Decide, per tool, which is more trusted.

Structured output and JSON schemas break. If you are doing structured outputs with JSON Schema, masking can produce a value that no longer satisfies its own field: a format: email field cannot hold [REDACTED EMAIL], and an integer field cannot hold asterisks. Length-preserving masking is friendlier because it keeps the shape. Masking the output leg is worse — the model produced valid JSON, the masker rewrote a value inside it, and your parser throws on a response the model got right.

Multi-turn consistency. Turn one: “Priya Raman is the account owner.” Turn four: “does she still own it?” If turn one became asterisks, the model has no thread to follow. Deterministic pseudonymisation fixes this, but only if the masker keeps state across turns — which a stateless guardrail does not.

RAG retrieval against masked text. Mask documents before indexing and the embedding no longer contains the entity, so a query for it will not retrieve the chunk. Index unmasked and mask at retrieval time, and your vector store holds the unmasked corpus — the thing you were avoiding.

Applied narrowly, masking is worth it. Applied globally, it quietly makes the product worse and nobody connects the two.

Where teams get this wrong

Assuming masking round-trips. A team masks PII on the way to the model, assumes something unmasks it coming back, and builds on that. Unless you chose a reversible scheme and something holds keys, the caller sees the masked value too.

Masking the prompt and forgetting the tool path. Data moves in four places: into the model, out of the model, into a tool, out of a tool. Covering one is common. The leak then happens through a tool result nobody was inspecting.

Treating log redaction as a data-flow control. Redacting stored logs limits who inside your company reads prompts. The provider still received the original text.

Choosing irreversible masking for a reversible problem. If something downstream needs the value back, masking is the wrong technique. You need tokenisation or FPE, which means a vault or a key.

How masking works in TrueFoundry

Masking is not a separate product here. Every guardrail carries two settings — Operation Mode and Enforcement Strategy — masking is a value of the first.

Mutate vs Validate

Priority is the whole ordering model for masking. Three mutate guardrails on one hook run in order, each seeing the output of the one before: a secrets masker at 1 and a regex masker at 2 means the regex guardrail matches text that already reads ***REDACTED***.

Input mutation is blocking and runs first — the model request cannot start until the prompt is final. Input validation then runs in the background alongside it, and if validation fails mid-flight the gateway cancels the model request so you do not pay for it. Output mutation and validation run synchronously after the response arrives, by which point the model cost is incurred.

MCP tool calls add two synchronous hooks: pre-invoke, where a failure means the tool never runs, and post-invoke, where a failure withholds the result. Guardrails run on every tool call separately — five tools, five sets of checks.

Enforce vs Enforce But Ignore On Error

The second setting separates two failures: the guardrail found something, versus the guardrail broke.

Strategy

On violation

On guardrail error

Enforce

Block

Block (fail-closed)

Enforce But Ignore On Error

Block

Let through (graceful degradation)

Audit

Let through, log only

Let through

The docs call Enforce But Ignore On Error “the safest default” and recommend the ladder Audit → Enforce But Ignore On Error → Enforce.

Guardrail enforcing strategy dropdown showing Enforce, Enforce But Ignore On Error and Audit

Which built-ins can actually mask

“We have a PII guardrail” tells you nothing about whether it can mask.

‍