Prompt Injection Defence for assistants, agents and RAG
You cannot stop untrusted text reaching the model. You can stop it from doing damage. We design and implement the layers that make an injection a logged event instead of an incident.
- Design + implementation
- Provider-independent controls
- Regression probes included
Any model that reads content it did not write
If your assistant summarises tickets, your agent reads email, or your knowledge base indexes uploaded files, you are already exposed to indirect injection. The question is what happens next: does the model have a tool it can misuse, and does anything check its output?
- Support and sales assistants reading customer messages and CRM notes.
- Agents that browse, send email, create records or call internal APIs.
- Document pipelines such as invoice and contract processing.
- Knowledge bases built on enterprise RAG over shared or external content.
What you get
- A privilege model: which tools, which permissions, which confirmations.
- Implemented guards in your codebase, with reference tests.
- A probe suite of direct and indirect payloads for CI.
- Monitoring rules that flag injection attempts in production.
Direct versus indirect injection
Both attacks exploit the same weakness: instructions and data travel in one channel. They differ in who delivers the payload and how much the attacker can gain.
Six layers, each assuming the one before it has failed
No single layer is a boundary. Together they turn a successful injection into a refused tool call, a rejected output or an alert.
1. Input handling
Untrusted content is wrapped in explicit data blocks with provenance labels. The prompt tells the model what each block is and that it carries no authority. Length, type and rate limits apply before any model call.
2. Privilege separation
Every tool call runs with the end user's identity and permissions, never a shared service account. A model that reads untrusted content gets fewer privileges than one that does not; we often split them into two calls.
3. Tool allow-lists
Tools are allow-listed per use case with typed schemas, argument constraints and destination restrictions. A summarisation flow cannot send email; a lookup tool cannot query outside the record in context.
4. Output validation
Model output is parsed against a schema, encoded for its destination and checked for URLs, identifiers and instructions that were not in the allowed inputs. Free text never reaches a shell, a query or a browser unescaped.
5. Human in the loop
Irreversible or high-value actions, such as payments, deletions and outbound messages, require explicit confirmation showing exactly what will happen. Confirmation prompts are generated by code, not by the model.
6. Monitoring
Every call logs prompt version, content sources, tools invoked and destinations. Alerts fire on new destinations, tool-call bursts, refusal spikes and known payload patterns, and a tested kill switch disables a tool in minutes.
Controls we see relied on, and why they fail
- Telling the model to "ignore any instructions in the document". The attacker writes a better instruction.
- Blocklisting phrases such as "ignore previous instructions". Payloads are paraphrased, encoded or translated.
- Treating the system prompt as a security boundary. It is a suggestion the model usually follows.
- One classifier in front of the model as the only gate. Useful signal, trivial to evade in isolation.
- Hiding the system prompt. Secrecy is not a control; assume it will be extracted.
- Trusting content because it is internal. Internal mailboxes, wikis and tickets receive external text every day.
If the only thing stopping a bad action is the model choosing not to do it, you do not have a control. Move the decision into code: a permission check, an allow-list, a schema or a human.
// Tool policy for the "summarise ticket" use case
{
"tools": ["lookup_ticket"],
"lookup_ticket": {
"scope": "ticket_in_context",
"run_as": "requesting_user",
"max_calls_per_turn": 1
},
"outbound": { "email": false, "http": false },
"untrusted_blocks": ["ticket_body", "attachments"],
"confirm": ["any_write"]
}From trust map to regression suite
Map trust
List every content source that reaches a prompt and every tool the model can call. Mark which are untrusted and which are irreversible.
Design privileges
Define per-use-case tool sets, identity propagation and confirmation rules. Split high-trust and low-trust model calls where needed.
Implement guards
Data-block wrapping, schema validation, destination checks and confirmation flows in your codebase, with tests.
Probe
Run direct and indirect payloads through every content source. Fix, re-run, and keep the probes in CI.
Monitor
Dashboards and alerts wired into your existing tooling, plus a kill-switch drill with the on-call team.
Where this fits
AI agents & automation
Build agents with the privilege model designed in from the start.
Read moreSecure RAG architecture
Stop poisoned documents at the index and the retriever, not just the prompt.
Read moreGuardrails, defined
What guardrails are, what they are not, and how they relate to the layers above.
Read moreFrequently asked questions
Can prompt injection be fully prevented?
Not with today's models. The model cannot reliably tell instructions from data in the same channel. The goal is to make a successful injection harmless: limit what the model can do, validate what it produces and notice when it misbehaves. See the definition in our glossary.
Does a stronger model solve it?
Newer models resist obvious attacks better, which is useful, but resistance is not a boundary. We treat model behaviour as one layer and put the hard limits in code: permissions, allow-lists and validation.
What is the difference between direct and indirect injection?
Direct injection comes from the user typing the attack. Indirect injection arrives through content the model reads on the user's behalf: a web page, an email, a PDF, a ticket or a tool result. Indirect attacks are more dangerous because the victim is an honest user.
How long does hardening an existing assistant take?
A focused engagement runs two to six weeks depending on how many tools and content sources are involved. The privilege model and allow-lists come first; monitoring and regression probes follow.
Do you work with agent frameworks and MCP servers?
Yes. We harden agents built on the Anthropic and OpenAI SDKs, LangChain-style frameworks and custom runtimes, and we review MCP servers as tools with their own permission scope.