Prompt injection is not a bug that gets patched. It is a structural consequence of how language models work: instructions and data arrive through the same channel, and the model has no reliable way to tell them apart. Every proposed input-filtering fix has been bypassed, usually within days. The teams shipping safely have stopped trying to solve it at the model layer and started containing it at the architecture layer.
01Why filtering does not work
The intuitive defence is to detect malicious instructions in input and block them. This fails because the space of ways to express an instruction is effectively infinite. Attackers use synonyms, other languages, encodings, unicode homoglyphs, indirection through fictional framing, and instructions split across multiple documents that only combine in context. A filter that catches today's phrasings is trivially bypassed by rephrasing.
It also fails in the other direction. Filters aggressive enough to catch real attacks reject legitimate content, and a security control that blocks real work gets disabled. If your product summarises emails, some emails will legitimately contain the word 'ignore' followed by instructions. You cannot filter your way out of ambiguity that is intrinsic to natural language.
02Indirect injection is the real threat
Direct injection - a user typing an attack into your chat box - is the version everyone thinks about and the least dangerous, because the user is attacking their own session. Indirect injection is where the damage lives: the model ingests content from somewhere else, and that content carries instructions.
A retrieved document, a web page the agent browses, an email in the inbox it summarises, a code comment in the repository it reviews, the alt text of an image, a calendar invite description. Any of these can carry text that the model reads as instruction. Now an attacker who has never touched your system can influence what it does on behalf of a legitimate user, with that user's permissions.
03Least privilege is the actual control
Since you cannot prevent the model being manipulated, limit what a manipulated model can do. This is the same principle that governs every other untrusted actor in your system, and it is the only defence that holds against attacks nobody has invented yet.
The agent's permissions should be scoped to the current user and the current task, not to the application's service account. If the agent can read any customer record, an injection can exfiltrate any customer record. If it can only read the record already in scope, the blast radius collapses to something you can reason about.
- Scope credentials to the requesting user, never a broad service identity.
- Separate read and write capability; require confirmation for writes.
- Whitelist egress destinations - injection needs a channel to exfiltrate through.
- Make destructive actions require an out-of-band human confirmation.
- Time-box and scope tokens to the single task in flight.
04Architectural patterns that hold up
Several patterns meaningfully reduce risk without depending on detecting the attack. They cost design effort rather than runtime filtering, and they degrade gracefully.
| Pattern | What it prevents | Cost |
|---|---|---|
| Dual-model isolation | Untrusted content reaching the privileged planner | Extra call; added latency |
| Structured tool output | Free-text smuggling instructions through results | Schema design and validation work |
| Human confirmation on writes | Silent destructive or irreversible actions | Friction on legitimate flows |
| Egress allow-listing | Data exfiltration to attacker-controlled endpoints | Maintenance of the allow-list |
| Per-user credential scoping | Cross-tenant and privilege-escalation impact | Auth plumbing through the agent layer |
05Treat model output as untrusted input
The mirror image of injection is what happens downstream. Model output frequently flows into a browser, a shell, a database query or another system, and it deserves exactly the same treatment as user-submitted input: validate against a schema, escape on output, parameterise queries, and never pass it to an interpreter.
Rendering model output as raw HTML is an XSS vector. Interpolating it into SQL is an injection vector. Passing it to a shell is remote code execution. These are ordinary vulnerabilities with a novel source, and the existing defences work perfectly well - teams simply forget to apply them because the output came from their own system.
06What to do on Monday
Start by enumerating every content source your models read and every capability they hold. Most teams find surprises in both lists - a tool added for a demo that still has write access, a retrieval source that ingests user-uploaded files. Map which sources can reach which capabilities, and that map is your threat model.
Then close the highest-leverage gaps first: scope credentials down, add confirmation to destructive actions, and restrict egress. Those three cover the majority of realistic attack paths, and none of them depend on predicting how an attacker will phrase things.
Topics
Marcus Hale
Lead Architect · SyncTrix
Writes about the engineering decisions behind production systems - architecture, delivery and the trade-offs that only show up at scale.
Building something like this?
SyncTrix engineers AI, SaaS, platform and cloud systems for enterprises and high-growth teams. Tell us what you're shipping and we'll scope it with you.