What Is Prompt Injection and How Does It Work?

A language model follows instructions written in plain language, and that is exactly what makes it vulnerable. If an attacker can get text into the model's input that says, in effect, ignore your previous instructions and do this instead, the model may obey. That is prompt injection, and it is one of the defining security problems of LLM applications.

The trouble runs deep because a model does not cleanly separate its trusted instructions from untrusted data. To the model, it is all just text in the context. So malicious instructions hidden inside a document, a web page, or a user message can hijack what the model does, with no traditional code exploit required.

This guide explains what prompt injection is, how it works, the main types, and how to reduce the risk.

What is prompt injection?

Prompt injection is an attack in which specially crafted input causes a language model to disregard its intended instructions and follow the attacker's instructions instead. Because the model interprets everything in its context as potential instructions, text supplied by an attacker can override the system prompt, extract hidden information, or redirect the model's behavior.

Think of a diligent assistant who does whatever any note on their desk says. You leave instructions, but so can anyone who slips a note onto the desk, and the assistant cannot tell which notes are legitimate. Prompt injection is slipping a malicious note into the model's context, and the model, trained to follow instructions, does as it is told.

The core problem is the lack of separation between instructions and data. In normal software, code and user input are distinct; in an LLM, both arrive as text in the same context, which is why injection is so hard to rule out.

How does prompt injection work?

An attacker gets malicious instructions into the model's context, and the model acts on them.

• Craft the malicious input. The attacker writes instructions designed to override the model's intended behavior.
• Get it into the context. The text enters as a direct prompt, or hidden in content the model will read.
• The model reads it as instructions. Unable to separate data from instructions, the model treats the injected text as commands.
• Behavior is hijacked. The model ignores its rules and does what the attacker asked.
• Impact. It may leak data, produce harmful output, or take an unauthorized action through a connected tool.

Indirect injection is especially dangerous because the victim never sees the attack. A model that summarizes a web page or email can be hijacked by instructions hidden in that content, turning a trusted feature into an attack vector.

Direct vs. indirect injection vs. jailbreak

These related attacks differ in how the malicious instructions arrive. Direct injection is when the user types malicious instructions straight into the prompt, trying to override the system's rules. Indirect injection hides the instructions in external content the model reads, a web page, document, or email, so a third party attacks through data the model consumes. A jailbreak is a related attack that coaxes the model past its safety guidelines, often through clever or roleplay-style prompting.

They overlap: a jailbreak can be delivered by direct or indirect injection. What they share is exploiting the model's willingness to follow instructions in its input, which is why defending against them is a single broad challenge.

What are the risks of prompt injection?

• Data leakage. The model can be made to reveal its system prompt or other sensitive context.
• Unauthorized actions. If the model can call tools, injection can trigger actions the attacker wants.
• Harmful output. It can be pushed to produce content it was instructed to avoid.
• Misinformation. A hijacked model can return manipulated or false answers to users.
• Trust erosion. Successful attacks undermine confidence in the application's safety.

How do you defend against prompt injection?

• Least privilege. Limit what tools and data the model can access, so a hijack can do little.
• Input and output guardrails. Scan for injection patterns and filter unsafe responses.
• Separate trust levels. Treat retrieved and user content as untrusted, not as instructions.
• Human approval. Require confirmation before high-impact actions a model might take.
• Defense in depth. Combine multiple controls, since no single one fully prevents injection.

When should you worry about prompt injection?

You should consider it for any LLM application that reads untrusted content or can take actions, which is most real applications. If your model summarizes web pages, processes user documents, or calls tools, injection is a live risk that must be designed against from the start.

A closed, offline model answering only trusted internal prompts faces less exposure. But as soon as external content or real-world actions are involved, prompt injection moves from a theoretical concern to a practical one.

Best practices for reducing prompt injection risk

• Give the model the least access to tools and data that the task requires.
• Treat all retrieved and user-supplied content as untrusted data, never as trusted instructions.
• Add input and output guardrails to detect injection attempts and unsafe outputs.
• Require human approval before any high-impact or irreversible action.
• Test your application against known injection techniques, not just the happy path.
• Enforce guardrails and access limits centrally at a gateway across all models.

Conclusion

Prompt injection is an attack that uses crafted input to make a language model ignore its instructions and follow the attacker's instead. It works because a model cannot cleanly tell trusted instructions from untrusted data, so malicious text, even hidden in content it reads, can hijack its behavior. It cannot be fully eliminated, but least privilege, guardrails, and treating external content as untrusted sharply reduce the risk.

TrueFoundry's AI Gateway lets teams apply input and output guardrails, control what models and tools can access, and monitor traffic for abuse, all in one layer. As LLM applications take on real content and actions, defending against prompt injection through a governed gateway is central to keeping them safe.

‍