Prompt injection: how to protect chatbots and AI agents (OWASP Top 10 for LLMs)

Prompt injection is an attack in which someone inserts malicious instructions into the text a language model reads, to make it do something it shouldn’t: reveal confidential information, ignore its rules, use a tool in a harmful way. It works because, to a model, instructions and data are the same text: there is no hard separation between “what it must do” and “what it is reading”. It is the number one risk in the OWASP Top 10 for LLM-based applications.

Direct and indirect prompt injection

• Direct: the attacker types the instructions into the chat. For example, they ask the website chatbot to “ignore the previous rules” and show its system prompt or a restricted discount code.
• Indirect: the instructions come from content the agent reads while it works: an email with hidden text, a web page, a PDF uploaded by a customer, a ticket, a review. The legitimate user sees nothing unusual, but the agent carries out the instructions.

The indirect form is the most dangerous because it targets precisely the most useful agents: those that read email, browse the web, consult documents and can take action.

Why it is the number one risk: the OWASP Top 10 for LLMs

The 2025 edition of the OWASP Top 10 for LLM Applications lists the ten main risks. Many of them are connected:

Code	Risk	In practice
LLM01	Prompt injection	Malicious instructions in the text change the model’s behaviour
LLM02	Sensitive information disclosure	The model reveals personal data, secrets or confidential information
LLM03	Supply chain	Compromised third-party models, libraries or data
LLM04	Data and model poisoning	Manipulated training or context data
LLM05	Improper output handling	The model’s output is used without checks, for example in a query or on a web page
LLM06	Excessive agency	The agent has more permissions or tools than it needs
LLM07	System prompt leakage	Internal instructions and secrets written into the prompt leak out
LLM08	Vector and embedding weaknesses	Uncontrolled access to the knowledge bases used for RAG
LLM09	Misinformation	False answers presented as true
LLM10	Unbounded consumption	Requests that send costs or load through the roof

Prompt injection is often the way in; excessive agency (LLM06) and improper output handling (LLM05) determine how much damage it can do.

Four concrete scenarios

1. The email assistant. An agent summarises the inbox and can send replies. An incoming email contains, in white text on a white background, an instruction to forward the latest invoices to an external address. If the agent is allowed to send emails without confirmation, it does exactly that.
2. The website chatbot. A user persuades the chatbot to show its system prompt, where someone had written an API key “for convenience”.
3. The agent with CRM access. A carefully crafted request leads the agent to extract the full customer list instead of just the record requested.
4. The poisoned knowledge base. A document uploaded to a RAG system contains instructions that, whenever the document is retrieved, change the answers given to every user.

Defences that work

No filter blocks every prompt injection. Effective protection is layered, and most of it sits outside the model.

1. Minimal permissions on tools

The agent should be able to do only what the task requires, with the permissions of the user it is working for. In the systems we design, the tools available to the agent depend on the role, and the check happens on every call. If an agent cannot export the customer list, no hidden instruction can make it export it.

2. Human confirmation for risky actions

Payments, messages to external recipients, deletions and bulk changes must go through explicit confirmation. The agent prepares, the person approves.

3. Separate instructions from data

External content should be passed to the model clearly delimited and marked as untrusted data. This is not enough on its own, but it greatly reduces the most basic attacks and makes checks easier.

4. Control what goes out

Many attacks aim to get data out: links to external domains, images that load a URL with the data in the query string, API calls. It is wise to allow only known domains, validate output before using it in other systems and never run generated code or queries without checking them.

5. No secrets in the prompt

Keys, passwords and confidential rules do not belong in the system prompt: sooner or later someone will get the model to repeat it. Secrets live in the systems the agent calls, protected by their own access controls.

6. Limits, logs and alerts

Per-user usage limits, a log of every action and alerts on anomalous behaviour make it possible to spot an attack and limit the damage, including the financial damage.

7. Red teaming before and after

An AI assistant should be deliberately attacked by someone who knows the techniques, before it goes live and after every change of model, prompt or tools.

How we test an AI assistant

In our AI system security tests, we follow a precise sequence:

1. Threat model: what the assistant can do, what data it sees, who can write to it and which sources it reads from.
2. Direct attacks: attempts to get around the rules, extract the system prompt, or make it say or do things it was never meant to.
3. Indirect attacks: documents, emails and pages prepared with hidden instructions and placed in the channels the agent reads.
4. Exfiltration and tool abuse: attempts to get data out or to use tools beyond their intended purpose.
5. Report: every vulnerability with its severity, evidence, impact and recommended fix, followed by a retest after the fixes.

Quick checklist

• Does the agent have only the tools and data that are essential to the task?
• Do permissions follow the user’s role, and are they checked on every call?
• Do irreversible actions require human confirmation?
• Is the system prompt free of secrets and confidential information?
• Is external content treated as untrusted data?
• Is output validated before it reaches other systems or a web page?
• Are links and images pointing to external domains restricted?
• Are there usage limits and alerts on anomalous behaviour?
• Is every action the agent takes logged?
• Was red teaming carried out before release?

If you are taking a chatbot or an AI agent into production, we can test it before someone else does.