Attacks on AI
Prompt injection defence: designing for a confusable tool
Prompt injection is the leading risk to LLM applications. The NCSC's advice is to stop hoping for a complete fix and to design so a fooled tool has little it can misuse.
Last reviewed against NCSC (UK), Anthropic and UK Government (DSIT and Home Office). Sources
The problem
Why it is not SQL injection
OWASP puts prompt injection first in its 2025 list of risks for LLM applications [9]. The NCSC blog of 8 December 2025 argues it should not be treated like SQL injection, because SQL injection can be fully prevented with parameterised queries while prompt injection probably cannot [2]. In the NCSC's words, LLMs do not enforce a security boundary between instructions and data inside a prompt [2].
Its suggested framing is an "inherently confusable deputy": a tool with real access that can be talked into using it for the wrong person [2].
Four measures
What the NCSC advises
Make people aware
Developers should know prompt injection exists, and security teams and risk owners should accept it as a residual risk that no product removes [2].
Design for failure
Use deterministic, non-LLM safeguards to limit what the system can do, and lower the LLM's privileges to match the least trusted source it processes [2].
Make attacks harder
Separate data from instructions where you can, but a blocklist of known attack wording is easy to get around and should not be your main protection [2].
Watch for probing
Keep records of what goes in and out, which tools ran and what APIs were called. Calls that fail can be a sign that someone is testing the system [2].
A test
Walk through one tool
Take any assistant you run and answer these in order:
-
1
What outside content does it read: email, web pages, uploaded files? -
2
What can it do with that content in view: send messages, change records, call APIs? -
3
Which of those actions would you refuse a stranger? -
4
Remove or require human approval for each one in answer three.
Be sceptical of promises
The NCSC says to beware any product that claims it can stop prompt injection [2]. If a use case cannot tolerate the residual risk, an LLM may not suit it [2].
More on using AI tools safely at work and whether the problem can be fixed.
Keep reading
Read next
AI tools at work
How UK organisations can govern the AI tools staff use, using NCSC guidance, the government's Code of Practice and the OWASP list of LLM risks.
Read the guide
Are they new?
Are AI powered attacks a new threat or a faster version of old ones? What the NCSC and the UK survey say, and what it means for priorities.
Read the guide
Can it be fixed?
The NCSC says prompt injection probably cannot be fully prevented. What that means for buying AI products and for how UK teams design them.
Read the guideHow CyPro can help
Need a hand with any of this?
Patching is one job. Keeping a vulnerable system safe until it is patched, and proving it, is several more. CyPro's consultants, testers and SOC analysts can take any of them off your plate.
Vulnerability scanning
Find every internet-facing system you run, including the forgotten ones, and see which are still on a vulnerable build.
See the scanning service
Hardening and segmentation
Our consultants put the controls in place that limit the damage when something is missed: firewall rules, network isolation and tighter admin access.
More on cypro.co.uk
24/7 security monitoring
Our SOC analysts watch your systems day and night for the signs of an attack in progress: unexpected processes, new outbound connections, logins that do not fit.
More on cypro.co.uk
Penetration testing
A focused test of your systems and the controls around them, so you know they hold before an attacker finds out for you.
More on cypro.co.uk
Incident response
If you may already be compromised, our responders contain it, preserve the evidence, find out what was touched and get you running again safely.
More on cypro.co.uk
Fractional CISO for the board
A senior security lead who advises the board on what to fix first, what to accept and what to tell customers, and owns the decision record.
More on cypro.co.uk
Know your exposure
Not sure you are protected?
Many of the risks here sit in how an organisation uses AI: what tools staff run, what they can reach and who owns the risk. AI governance work sets that out so controls match the tools in use.