Prompt Injection: Securing Your AI Assistant

When an AI chat assistant accepts questions from visitors, reads content from your own website and triggers connected actions, it constantly processes text from outside. That is exactly the entry point for prompt injection: the attempt to slip new instructions to the assistant through cleverly worded inputs or hidden snippets of text. The OWASP Foundation lists prompt injection in the current 2026 edition of its Top 10 for LLM applications as LLM01 and therefore in first place (OWASP), as it already did in the 2025 edition (OWASP). The reason is structural: a language model processes data and instructions in the same channel, with no clear separation between what it should do and what it should only read. This article explains, factually, what distinguishes direct from indirect attacks, why prompt injection cannot be switched off with a single filter, and which guardrails make an assistant resilient: input filtering, output checks, no secrets in the prompt, clearly limited permissions and vigilant monitoring. It deliberately sets the topic apart from data protection, because this is not about the GDPR but about attack security. No one can promise absolute security, but a considered concept lowers the risk considerably.

Enlarge graphic

Labels in the illustration

• Direct injection
• Instruction in the input field
• Indirect injection
• Hidden in website content
• Guardrails
• Input filtering
• Output checks
• No secrets in prompt
• Monitoring & logging
• Defense in depth
• OWASP rank 1
• Protected assistant
• Foreign instruction flagged
• Only approved actions
• No secrets in context
• Anomalies are logged
• Why hardening matters
• 25%
• GenAI apps with incidents
• per year by 2028 (Gartner)
• 9%
• 2025 baseline of GenAI
• security incidents (Gartner)
• 57%
• of companies in Germany
• now use AI (Bitkom)
• Separate data from instructions, add guardrails, monitor anomalies

Securing Against Prompt Injection

Why Prompt Injection Sits at the Top

Prompt injection is not an exotic edge case but the most widely recognised risk of generative AI. In the current OWASP Top 10 for LLM applications from 2026, prompt injection stands as LLM01 at the top of the ten risk categories in total (OWASP), followed by topics such as sensitive information disclosure as LLM02 and the exposure of hidden context such as the system prompt (hidden context exposure) as LLM08 (OWASP), a category that the 2025 edition still called system prompt leakage. There is a simple reason why manipulation through text of all things sits at the top: it affects every application that accepts text from outside at all, and a chat assistant does exactly that by its very nature. NIST too describes prompt injection in its taxonomy of adversarial machine learning, NIST AI 100-2e2025 from March 2025, as its own class of attack and there explicitly distinguishes direct from indirect variants (NIST).

For a chat assistant this is not an academic question. It is precisely the kind of application that continuously takes in text from outside: questions in the chat window, content from your own website, details from a form, sometimes data from connected systems. Each of these sources can, intentionally or not, contain text that looks like an instruction. How many companies this risk now concerns is shown by adoption: in Germany, 57 percent (Bitkom) of companies with 20 or more employees now use AI, up from 36 percent (Bitkom) the year before. As the number of live assistants grows, so does the attack surface, which is why security belongs in the picture from the start.

Data and Instructions in the Same Channel

The actual cause runs deeper than a single programming error. A language model receives its system prompt, the conversation history and the user's input as one continuous stream of text. Technically there is no hard boundary that says: up to here are binding instructions, from here on there is only material to read. To the model, both are language. Germany's Federal Office for Information Security pointed out as early as 2023, in a cyber-security warning, that indirect prompt injections are an intrinsic weakness of application-integrated AI language models, so not a passing teething trouble but a property of the technology (BSI).

Because the weakness is structural, it cannot be switched off with a single filter. Both NIST and the BSI therefore recommend not a single line of defence but a layered approach of several complementary measures, often called defense in depth (NIST, BSI). The aim is not to rule out every conceivable attack, but to reduce the likelihood and the possible impact far enough that an assistant can go live responsibly. This thinking in layers runs through the sections that follow.

Direct and Indirect Attacks

The taxonomy from OWASP and NIST distinguishes two basic forms that differ not in their goal but in their path (OWASP, NIST). Both want to change the assistant's behaviour, but one arrives directly through the input field while the other hides in content the assistant merely reads.

The indirect variant is especially treacherous, because the instruction does not come from the visible conversation partner but from a seemingly harmless source. NIST added exactly this extension to its taxonomy AI 100-2e2025 and describes indirect prompt injection as an attack in which prepared content is planted in data sources that a system later reads and processes (NIST). For an assistant this means: every external source it draws knowledge from should be treated as potentially manipulated and must not be executed like a command unchecked. How an assistant instead ties itself closely to checked, curated knowledge is therefore not only a question of answer quality but also of security.

What a Successful Attack Causes

If a prompt injection succeeds, the consequences range from harmless to sensitive, depending on which capabilities and access the assistant has. The OWASP list assigns several of these consequences to their own categories, which shows how closely the risks are connected (OWASP). The typical effects are, above all:

• Disclosure of information held in the system prompt or context, which corresponds to the hidden context exposure category as LLM08, called system prompt leakage in the 2025 edition (OWASP)
• Bypassing the defined rules, limits and tone, often referred to as a jailbreak
• Triggering unwanted actions through connected tools, which OWASP lists as excessive agency, LLM03 (OWASP)
• Output of false, distorted or manipulated information to visitors
• Passing manipulated text on to downstream systems, where it can cause further harm

That such incidents are not mere theory is shown by the outlook: Gartner expects that by 2028 25 percent (Gartner) of all enterprise applications using generative AI will record at least five minor security incidents per year, up from 9 percent (Gartner) in 2025. For serious incidents, Gartner anticipates a rise to 15 percent (Gartner) by 2029, up from 3 percent (Gartner) in 2025. And by 2028, 50 percent (Gartner) of enterprise incident response efforts are expected to focus on incidents involving custom-built AI-driven applications. This direction underlines that hardening is not a one-off hurdle but part of ongoing operation.

Attack Security Is Not Data Protection

Prompt injection security and data protection are often lumped together, yet they are two different building sites. Data protection under the GDPR is about processing personal data lawfully, sparingly and transparently, with a legal basis, a data-processing agreement and a deletion concept. Attack security is about something else: protection against targeted manipulation, that is, making sure the assistant is not lured into unwanted behaviour by injected text. An assistant can be hosted in exemplary compliance with the GDPR and still remain vulnerable to prompt injection, and vice versa. Both topics belong together in thought but must not be confused. What matters concretely for data protection is covered in the article on the GDPR-compliant AI assistant; the question of labelling AI in the chat, in turn, is the subject of the EU AI Act and the transparency obligation.

Guardrails That Protect

Because there is no single switch, robust hardening relies on several layers that work together. Both the BSI and OWASP recommend a combination of technical and organisational measures rather than a single silver bullet (BSI, OWASP). The following six guardrails have proven their worth in practice (project experience) and interlock, so that an attack would have to clear several hurdles at once.

Two of these guardrails reach especially deep into how an assistant is built. That it only carries out clearly defined, approved actions and does not receive blanket full access to connected systems is the core of controlling tools via function calling and is described on the page about tool control with clear permissions. That it ties its answers closely to checked content instead of formulating freely is shown in the article on how to prevent hallucinations with a knowledge base and on the page about the knowledge base. Both building blocks improve answer quality and security at the same time.

Unprotected or Secured: the Comparison

The difference between an assistant without guardrails and a secured assistant is not cosmetic but touches every layer of processing. The following comparison sums up how the two approaches differ.

As important as these guardrails are, so is the honest framing: even a secured assistant does not become unassailable. NIST states explicitly that, as things stand, there is no foolproof protection against prompt injection, because the weakness is rooted in how the models work (NIST). The aim is therefore a clear reduction of risk, paired with a clear fallback line: where something is unclear or becomes sensitive, the assistant is meant to hand over to a human instead of guessing. How a clean handover to a human agent with full context works is described in the corresponding article.

How XICBOT Secures an Assistant

At XICBOT, hardening is part of building an assistant, not a later add-on. An individually trained assistant has an advantage here over an arbitrary off-the-shelf chatbot, because its rules, permissions and limits can be set deliberately; where the differences lie is shown in the article on the individually trained assistant versus a standard chatbot. The technical integration into your website cleanly separates the necessary functions, and the data is kept in hosting in Germany. Even where an assistant becomes active and, for example, recovers abandoned carts in the chat, the same principle applies: clearly limited actions instead of blanket access.

• Are inputs filtered and checked for suspicious patterns?
• Is external content read as material rather than executed as an instruction?
• Is it ruled out that secrets such as keys or internal data sit in the system prompt?
• Does the assistant receive only clearly defined, approved actions instead of full access?
• Are outputs checked before they are passed on to visitors or systems?
• Are anomalies logged, evaluated and the rules refined accordingly?
• Is a clean handover to a human provided for when something is unclear?

That security is becoming standard for AI applications is emerging: by 2028, more than 50 percent (Gartner) of companies are expected to use dedicated AI security platforms to secure AI applications. At the same time, in the 2026 Bitkom survey, 56 percent (Bitkom) of companies in Germany already using AI name legal uncertainty as one of the biggest obstacles. An assistant with well-considered guardrails takes away part of that uncertainty, because it shows transparently which measures are in place. Which safeguards suit your use case is best clarified together in an initial consultation via the contact page. No one can promise absolute security here, but a transparent, layered concept can be provided.

Frequent questions