Skip to content
SI.info

Prompt Injection

Published 2 min read
On this page
  1. Direct and indirect prompt injection
  2. Why it matters for SI agents
  3. Where the name comes from
  4. Defenses

Prompt injection exploits a basic weakness of large language models: they read instructions and data as one stream of text. If attackers can get their own text in front of a model, they can try to give it orders, such as “ignore your previous instructions and…”.

Direct and indirect prompt injection

  • Direct: the attacker types the malicious instructions into the app themselves. OWASP describes this as a user’s prompt that “directly alters the behavior of the model”.
  • Indirect: the instructions hide in content the model processes for someone else, such as a web page, an email, a PDF, a code comment or the result of a tool call. OWASP describes this as a model that “accepts input from external sources, such as websites or files”. The hidden text can even be invisible to the human user.

Why it matters for SI agents

An injected chatbot might say something embarrassing. An SI agent that can read email, browse the web, edit files or make payments could be tricked into leaking data or taking harmful actions on the attacker’s behalf. That is why prompt injection is number one (LLM01) in the OWASP Top 10 for LLM Applications 2025, and why it is a central concern for computer use and tool standards such as the Model Context Protocol.

Where the name comes from

In September 2022, developer Simon Willison wrote about attacks on apps built with GPT-3, first demonstrated by Riley Goodside, in which users told the model to ignore its instructions. “I propose that the obvious name for this should be prompt injection,” he wrote, adding: “The obvious parallel here is SQL injection.”

Defenses

OWASP’s recommended mitigations include:

  • constraining what the model is allowed to do and validating its output format;
  • filtering inputs and outputs;
  • giving the app and its agents the least privilege they need;
  • requiring human approval for high-risk actions;
  • clearly separating and labelling untrusted external content;
  • adversarial testing, also known as red teaming.

None of these is a complete fix on its own, so developers layer them.

← Back to the SI Glossary

Frequently asked questions

What is the difference between prompt injection and a jailbreak?

A jailbreak is a user trying to talk a model out of its own safety rules. Prompt injection is someone else's text, hidden in content the model reads, overriding the instructions of the user or developer. The tricks often overlap, but prompt injection is mainly a security problem for apps and agents.

Can prompt injection be completely prevented?

Not reliably. The OWASP Top 10 for LLM Applications (2025) says “it is unclear if there are fool-proof methods of prevention for prompt injection.” Defenses reduce the risk, which is why developers limit what agents can do and require human approval for high-risk actions.

Sources

  1. Prompt injection attacks against GPT-3 — Simon Willison, September 12, 2022
  2. LLM01:2025 Prompt Injection — OWASP Gen AI Security Project, 2025

Written by

· Editor

Editor of SI.info. Writes about Super Intelligence, technology policy and the people building frontier models.

How we research and fact-check

Free newsletter

Get The SI Brief

One short email a week: what changed in Super Intelligence, policy and models — and why it matters.

Free. One email a week. Sent via beehiiv, which counts opens and clicks. Unsubscribe anytime. Privacy policy