The InfoSec community is strongest when it can collaborate openly. Few organizations can fend off sophisticated attacks alone—and even they sometimes fail. If we all had to independently discover every malware variant, every vulnerability, every best practice, we wouldn’t get very far. Over time, shared mechanisms emerged: VirusTotal and EDR for malware, CVE for vulnerabilities, standard organizations like OWASP for best practices.

It’s remarkable that we can rely on one another at all. Companies compete fiercely. But their security teams collaborate. That kind of collaboration is not trivial. Nuclear arsenals. Ever-growing marketing budgets. These are places where more collaboration could help—but doesn’t.

So when attackers hijack AI agents through prompt injection, we fall back on what we know: vulnerability disclosure. Vendors launch responsible disclosure programs (which we applaud). They ask researchers to report successful prompt injections privately. The implicit implication: responsible behavior means private reporting, not public sharing.

But there’s a problem, prompt injection can’t be fixed.

Blocking a specific prompt does little to protect users. It creates an illusion of security that leaves users exposed. Ask any AI security researcher: another prompt will surface—often the same day the last one was blocked.

Calling a vulnerability “fixed” numbs defenders, hiding the deeper issue. Anything your agent can do for you, an attacker can do too—once they take control. 

It’s not a bug, it’s a design choice. A tradeoff.

Every AI agent sits somewhere on a spectrum between power and risk. Defenders deserve to know where on that spectrum they are. Open-ended agents like OpenAI Operator and Claude’s Computer Use maximize power, to be used at one’s own peril. AI assistants make different tradeoffs exemplified by their approach to web browsing—each vendor has come up with their own imperfect defense mechanism. Vendors make choices which users are forced to live with.

Prompt injections illustrate those choices. They are not vulnerabilities to be patched. They’re demonstrations. We can’t expect organizations to make informed choices about AI risk without showing visceral examples of what could go wrong. And we can’t expect vendors to hold off powerful capabilities in the name of safety without public scrutiny.

That doesn’t mean vulnerability disclosure has no place in AI. Vendors can make risky choices without realizing it. Mistakes happen. Disclosures should be about the agent’s architecture, not a specific prompt.

Rather than treating prompt injection like a vulnerability, treat it like malware. Malware is an inherent risk of general-purpose operating systems. We don’t treat new malware variants as vulnerabilities. We don’t privately report them to Microsoft or Apple. We publicly share it, as soon as possible, or rather its hash. We don’t claim that malware is “fixed” because one hash made it onto a denylist.

When a vulnerability can be fixed, disclosure helps. But when the risk is inherent to the technology, hiding it only robs users of informed choice.

Naming is powerful. An excellent name does more than frame the problem, it hints at ownership, solutions, and urgency to address it. In a very real sense, they are like equivalence proofs in mathematics–allowing us to apply knowledge from one domain to the other.

When Simon Willison’s prompt injection term got widely accepted, it changed reality for both the AI and security communities. We’re still living in the reality this term has created. But we are also confined by it, unable to mentally escape its comforting prediction.

AI researchers now had a fresh perspective they could leverage and learn from–that of trusted vs contaminated data. The solution of separating data from instructions follows, with excellent work like Instruction-Hierarchy and CaMeL.

Security practitioners had a wealth of experience they could now apply. The hard lessons of SQL injection roared back from the late ’90s. Many of our most senior security leaders have lived through it. They did not need convincing that this is an important problem, nor that this is their problem to solve (with developers). And they knew the solution: establish vulnerability disclosure programs, perform taint analysis to proactively identify issues, advocate for separation of data from instructions.

SQL-injection is a solved problem today, at least in the technical sense–use the right library, don’t use eval-like functions, and you’re covered. Surely, AI labs will figure out a solution to prompt injection down the line. Right?

We have reaped the benefits of the term prompt injection, and we are much better for it. But its ubiquitousness over time can limit our perspective.

We’ve started to conflate The Problem with prompt injection. But prompt injection is not the entire problem. The problem is that AI inherently does not follow instructions, and we act like it does. It follows our goals, an attacker’s goals, or its own internal goals all the same. Attackers exploit this to nudge your AI into accomplishing their goals. So far we know they can do it through prompt injection directly and indirectly by manipulating its environment, and very indirectly planting traps in its training data.

It’s not bad implementation of a particular AI agent. Or a specific model that is just too damn susceptible to attacks (they all are). Faulty system instructions (fancy name for wrapped section of the prompt) are not it either. The entire premise is flawed.

The term prompt injection makes us imagine malicious content coming in and tainting our well-behaved AI. But AI is trained on the internet, remember? The same internet we use, with its digital skyscrapers, dirty gutters and flourishing undergrounds. We’ve become hyper-focused on building a security perimeter around AI (another echo from the ’90s), and forgot to look up and stare at the damned thing–AI is tainted to its core.

I was tempted to end this piece without a suggested alternative. Feeling it would dilute my argument. But making progress requires taking a step forward.

Back at BlackHat USA 2024 I suggested the Remote Copilot Execution (the term Copilot has lost favour since). It has the clear benefit of drawing a parallel to Remote Code Execution, driving up urgency and emphasizing its open-endedness. But the problem remains–RCEs are something bad that happens so well-behaved AI.

I suggest the term AI Hijacking, or AIjacking. It places the emphasis on the outcome we’d like to avoid–attackers hijacking our AI–rather than how they do it.

Hijacking our AI means an attacker can use its access and tools at their disposal. It puts the emphasis on agentic AI. An attacker only needs to hijack the agent’s goal, and the agent figures out how to best use its capabilities to wreak havoc.

This is a boring blog post. At least for humans.

Allowing a copilot to search the web at will is extremely dangerous. Here are two somewhat-understood vulnerabilities and how to mitigate them. Note: this is an ever-evolving field and this is only what I know today. Who know what you’ll know tomorrow!

The first vuln is prompt injection.

You should never blindly trust things written on the Internet, right? Well, GenAI is happy to follow any instructions, if you write your them well. A copilot with Internet access is always one search away from a website with hidden malicious instructions, getting taken over by an external attacker.

The second vuln is data exfiltration.

You know who else is on the Internet? Everyone else, including them. If an attacker can compromise your copilot, they can instruct it to search a website they control. That means copilot will reach out - proactively, without a user in the loop - to a website a hacker controls. The common exploit is encoding data to-be-exfiltrated into a parameter.

We do have some design patterns to address these. Apply one of the following:

1. You could limit which website copilot can search. See Content Security Policy (employed by Microsoft Copilot and Google Gemini) and URL Anchoring (employed by ChatPT).

2. You could let copilot ask a trusted third party to look at the website and provide key information. See Index-Based Browsing Security Policy.

3. You can decide its too risky and remove web browsing entirely.

Assorted links for All You Need Is Guest @ RSAC 2024:

1. Power Platform DLP Bypass via Copy & Paste
2. OWASP No-Code / Low-Code Top 10
3. powerpwn
4. Microsoft docs on EntraID multitenant sharing options

Other talks (slides and source code)

1. All You Need is Guest @ BlackHat USA 2023
2. Sure, Let Business Users Build Their Own. What Could Go Wrong? @ BlackHAt USA 2023
3. Low Code High Risk: Enterprise Domination via Low Code Abuse @ DEFCON30
4. No-Code Malware: Windows 11 At Your Service @ DEFCON30