Secrets Detection in Prompts: Stop Credential Leaks to LLMs

Why secrets end up in prompts

Nobody means to send a credential to a model provider. It happens anyway, because prompts are assembled from messy, real-world text:

A developer debugging an integration pastes a config snippet — with a live API key in it — and asks the model to find the bug. A coding agent, told to “fix the failing test,” reads the repo’s .env file into its context. A RAG pipeline retrieves a document that happens to contain a database connection string. An agent calls a tool that returns an object with an embedded OAuth token, and that result goes straight back into the model’s context.

In every case the secret has now been transmitted to a third-party provider, logged somewhere, and possibly used in training or retained. The developer did nothing obviously wrong; the system had no checkpoint. That checkpoint is what a secrets-detection guardrail provides — and the right place for it is the gateway every call already passes through.

What leaks, and where

Credentials come in recognizable shapes, and a good detector knows the patterns.

TrueFoundry’s built-in Secrets Detection guardrail recognizes 40-plus credential patterns across cloud providers (AWS, Azure, GCP), AI/ML API keys (OpenAI’s sk-, Anthropic’s sk-ant-), repository and package tokens (GitHub, GitLab, NPM, PyPI), communication platforms (Slack, Discord, Telegram), payment keys (Stripe, Square, Shopify), database connection strings, and private keys and certificates (RSA, OpenSSH, PGP). Beyond fixed patterns, it does contextual detection — flagging high-entropy hex or base64 strings that appear near keywords like token=, secret=, or password=, which catches credentials that don’t follow a standard format.

Because it runs inside the gateway rather than calling an external service, detection is fast and adds no extra per-request cost.

Where to catch it: four hooks

A secret can enter the conversation from three directions — the user, the model, or a tool — so the guardrail runs at four points.

LLM Input stops a user or app from sending credentials to the model in the first place. LLM Output catches a model that generates or echoes a secret (for example, producing a sample config with a real-looking key). MCP Pre-Tool scans tool parameters before a tool runs, and MCP Post-Tool redacts secrets in tool results — the file contents and API responses an agent pulls back into its context. For agentic systems, the MCP hooks matter as much as the LLM ones, because the agent is reading files and calling APIs on its own.

Block or redact

You choose what happens when a secret is found.

Validate mode blocks the request outright when a secret is detected — the right choice when a credential in the content is a hard policy violation. Mutate mode redacts the secret, replacing it with configurable text (***REDACTED*** by default) and letting the request continue — useful when you want the agent to keep working on the non-sensitive part of the content, and when you want clean text in your logs. Many teams run Mutate on input (so work continues with the secret stripped) and consider Validate where a leak must hard-stop.

Where teams get this wrong

They only guard user input. Secrets leak from model output and tool results too. Guard all four hooks, not just the prompt.

They forget output guardrails skip streaming. With stream: true, output scanning doesn’t run — there’s no complete response to check. Use stream: false where output redaction matters.

They rely on developers to be careful. “Don’t paste secrets” is not a control. The whole point of a gateway guardrail is that it doesn’t depend on anyone remembering.

They don’t tune for false positives. High-entropy strings like UUIDs or encoded blobs can trip detection. Test against your real traffic and adjust before enforcing hard blocks.

Want credential leaks stopped before they reach a provider?

Book a demo

How to turn on secrets detection in TrueFoundry

It’s a built-in guardrail, so there are no external keys to configure. In the AI Gateway, open Guardrails, create or pick a group, and add the Secrets Detection guardrail.

Configure the name, the operation (validate or mutate), the enforcing strategy, and — for mutate — the redaction text. Then apply it per request with the X-TFY-GUARDRAILS header:

{
  "llm_input_guardrails": ["my-group/secrets-detection"],
  "llm_output_guardrails": ["my-group/secrets-detection"]
}

Or enforce it org-wide, including on MCP tools, with a gateway policy:

name: guardrails-control
type: gateway-guardrails-config
rules:
  - id: mcp-secrets-protection
    when:
      target:
        conditions:
          mcpServers: {values: [file-reader], condition: in}
      subjects:
        conditions:
          in: [team:engineering]
    mcp_tool_post_invoke_guardrails:
      - my-group/secrets-detection

For maximum coverage, attach it to all four hooks — input, output, pre-tool, and post-tool.

A worked example: a coding agent reads an .env file

An engineering team runs a coding agent that can read repository files through an MCP file tool. A developer asks it to “fix the failing database test.” The agent reads config/.env to understand the connection — and that file contains postgresql://admin:S3cr3tP@ss@db.prod:5432/app.

Without a guardrail, that connection string goes into the model’s context and out to the provider. With Secrets Detection on the MCP Post-Tool hook in Mutate mode, the tool result comes back as postgresql://admin:***REDACTED***@db.prod:5432/app before the model ever sees it. The agent still gets the structure it needs to reason about the test; the live credential never leaves the environment. Add the LLM Input hook and the same protection covers the developer pasting the string directly into chat.

Gotchas worth knowing

• Cover all four hooks. Input-only misses secrets from model output and tool results.
• Streaming skips output scanning. Set stream: false when you need output redaction.
• System prompts aren’t scanned by default — the gateway strips them before guardrails run.
• Watch false positives. UUIDs and encoded data can look like secrets; test before enforcing.
• Prefer Mutate for flow, Validate for hard stops. Redaction keeps agents working; blocking is for zero-tolerance cases.

Related reading

• TrueFoundry AI Gateway guardrails, explained
• Configure guardrails on the AI Gateway
• PII redaction: LLM gateway vs application
• Prompt injection defense at the LLM gateway
• Enterprise AI security with the MCP Gateway: runtime guardrails
• What is AI governance?

Conclusion

Secrets get into prompts the same way they get into logs — not through malice, but through ordinary work hitting a system with no checkpoint. A coding agent reading a config file, a developer pasting a stack trace, a tool returning a token: each is a credential one hop from a third-party provider.

A secrets-detection guardrail at the gateway is that checkpoint. It scans every hook, recognizes the credentials that matter, and either blocks or redacts them before they leave your perimeter — fast, in-gateway, and not dependent on anyone remembering the rule. It’s one of the cheapest, highest-leverage guardrails you can turn on.

Stop secrets leaking into prompts

Book a demo