An AI assistant that gives customers information is only as reliable as what it bases its answers on. This is exactly where the biggest misunderstanding about generative language models lies: they are trained to produce the most probable next sentence, not to know the truth. Without a link to checked content, a model can phrase a wrong answer just as confidently as a correct one. In technical terms this is called hallucination or, as the NIST AI Risk Management Framework puts it, confabulation. A survey shows how seriously companies take the topic: around 44 percent (McKinsey) of organizations report at least one negative consequence from generative AI, and nearly a quarter (McKinsey) of those cases trace specifically to inaccurate output. Even specialised research tools linked to expert databases get it wrong: in a study of legal AI tools, the hallucination rate ranged between 17 and 33 percent (Stanford HAI) despite source grounding. This article explains why a generic model invents facts, and how a well-maintained knowledge base with retrieval-augmented generation binds answers to your real content: through source binding, confidence thresholds with escalation to humans, honest handling of knowledge gaps and content freshness.
Why a Language Model Invents Facts
A large language model is at its core a probability calculator for language. It has learned from a great many texts which words usually follow one another, and it assembles an answer word by word from what fits best statistically. That explains why the outputs sound so fluent and convincing. But it also explains the underlying problem: the model has no built-in concept of whether a statement is true. It does not distinguish between evidenced knowledge and a plausible-sounding invention, but produces a well-phrased sentence in both cases. When a sought piece of information is missing, the model often fills the gap with something that looks fitting instead of admitting the gap openly.
In customer contact this turns into a tangible risk. An assistant not bound to its own content can quote a price that does not exist, invent a return period, describe a product that is not in the range or make a commitment no one intended. Precisely because the wrong answer is phrased just as confidently as a correct one, the error is not immediately obvious to the person asking or to the business. What an AI assistant fundamentally does and where its limits lie is set out in our article on what an AI chat assistant is. The decisive lever against invented facts is to give the model a checked foundation to answer from.
Briefly explained: hallucination and confabulation
What a Knowledge Base With RAG Changes
Retrieval-augmented generation, RAG for short, connects the language model with a searchable knowledge base. Instead of forming an answer from general training knowledge alone, the system first retrieves the relevant passages from your content and lets the model phrase the answer on this retrieved basis. The approach goes back to the work of Lewis and colleagues at Meta AI Research, who showed in 2020 that such a method reached new best results on demanding knowledge tasks, including 44.5 points exact match on open-domain question answering with Natural Questions (Meta AI Research). The heart of the principle: the model does not have to memorise facts, it looks them up in a reliable source at the moment of the question.
For answer quality, the decisive finding of the same work is even more telling. Human reviewers rated the source-grounded answers as more factually correct in 42.7 percent of cases, against only 7.1 percent for a comparable model without a link (Meta AI Research). The retrieved answers were also judged more concrete and specific far more often, in 37.4 percent versus 16.8 percent of cases (Meta AI Research). Binding answers to checked content therefore not only reduces invented statements, it also makes the answers more precise. How we build and maintain this foundation for your assistant is described on the page about the knowledge base.
Lookup instead of memorising
The system retrieves the passages relevant to the question from your content and answers from them, instead of relying on a rigid memory frozen during training.
Answer with context
The retrieved passages provide the frame in which the model phrases its reply. That keeps the answer bound to your specific offer rather than general world knowledge.
Updatable without retraining
When a piece of content changes, the source is updated and the assistant answers accordingly, without a whole model having to be retrained.
Source Binding: Every Answer With Evidence
Source binding means the assistant derives its statements from your company's approved content and not from the model's general world knowledge: from the website, the product catalog, the price list, the FAQ and checked documents. What is not written there is not claimed. Ideally the assistant can also name the origin of an answer, such as a particular FAQ or the catalog entry, so the information can be traced and, in case of doubt, checked. A statement you have to trust thus becomes a statement you can evidence.
Answers you can verify
This binding is not a nice-to-have but a recognised safeguard. The NIST AI Risk Management Framework lists confabulation as one of twelve risk categories for generative AI and sets out more than 200 recommended actions for managing such risks (NIST AI Risk Management Framework). Binding to checked sources is among the most effective of them. At XICBOT we define per project how closely answers are coupled to your content and which areas the assistant may answer at all. How it executes only clearly defined, approved actions instead of acting on its own is shown in the article on tool control.
Confidence Thresholds and Escalation to Humans
Even with a knowledge base a residual uncertainty remains: sometimes the retrieval finds no well-matching passage, sometimes the question is ambiguous, sometimes two sources contradict each other. Confidence thresholds exist precisely for this. Put simply, the system assesses how well the retrieved content fits the question and how robust an answer from it would be. If that value is above a set threshold, the assistant answers from the source. If it is below, it deliberately does not answer on a hunch but passes the concern on. A silent guessing game becomes a controlled decision.
Better to defer than to guess
On handover, the full conversation context goes with it to a member of staff, a contact form or a callback, so no one has to ask their question twice. That is not an admission of weakness but a quality feature: the assistant knows its limits and does not overstep them. It is no coincidence that inaccurate output is, according to a survey, the most commonly cited risk when using generative AI (McKinsey). How a clean handover works around the clock without overstepping professional limits is described in the article on the support assistant.
Admitting Knowledge Gaps Instead of Guessing
An honest I don't know is worth more in customer contact than a confident false statement. A well-configured assistant therefore treats a knowledge gap not as a flaw it must cover up, but as a cue to offer the next helpful step. Instead of inventing a return period, it says that it has no confirmed detail on that and offers to pass the question to the team or arrange a callback. The person asking is not left with a possibly wrong answer but on a reliable path to the right one.
The most reliable answer to an open question is not the fastest invention, but an honest note about the gap and the way to a human who can close it.
| Situation | Generic model without binding | Source-bound XICBOT assistant |
|---|---|---|
| Question outside the content | Phrases a plausible-sounding guess | Points out the knowledge gap and defers |
| Price or deadline unclear | Quotes an invented value | Answers only from a stored source or asks back |
| Conflicting information | Picks a variant at random | Flags the uncertainty instead of hiding it |
| Sensitive topic | Answers without safeguards | Deliberately hands over to a human |
| Origin of the answer | Not traceable | Traceable to a nameable source |
Which questions frequently run into a void is itself valuable information. Every recognised knowledge gap shows where content is missing or unclearly worded. The anonymized conversation analytics make these patterns visible, so the knowledge base can be extended precisely where visitors actually look for answers. A gap thus becomes a concrete maintenance task instead of a silent source of error.
Freshness: a Knowledge Base That Grows With You
Even a perfectly bound answer is worthless if the underlying source is out of date. Yesterday's price, an expired promotion or changed opening hours are otherwise reproduced correctly from the source and are still wrong. Here RAG plays its second big strength: because the retrievable knowledge sits separately from the language model, it can be updated without retraining the model (Meta AI Research). You maintain the content in one place, and the assistant answers on the new state from then on.
In practice this means regularly syncing the knowledge base with the living sources: the website, the shop catalog, the prices and the documents. Equally important is the counterpart, namely sorting out outdated content so the assistant does not answer from an old version. At XICBOT this sync, the maintenance and hosting in Germany are part of the ongoing service, so the foundation stays current without tying up internal resources. How the assistant connects technically to your existing systems is described in the article on integration.
- Sync website, shop and documents with the knowledge base regularly
- Actively sort out outdated content instead of leaving it in place
- Treat prices, deadlines and availability as especially time-critical
- Close recognised knowledge gaps from the conversation analytics on purpose
- Set firm limits and a handover to humans for delicate areas
- Spot-check answers against the stored source
Limits: Not a Self-Runner Even With a Knowledge Base
A knowledge base with source binding considerably lowers the risk of invented facts, but it does not remove it entirely. That can be read from the study already mentioned: even specialised research tools linked to expert databases returned a wrong or poorly supported answer in at least one in six demanding queries (Stanford HAI). The causes are known: the retrieval finds the wrong passage, a question is ambiguous, two sources contradict each other, or the content itself contains an error. Anyone promising that a system is thereby infallible misreads the technology.
No system is infallible
Good results therefore arise from the interplay of several precautions: cleanly maintained sources, a clearly delimited area of responsibility, ongoing observation of the conversations and human oversight for delicate cases. Reliability also includes transparency towards users. Why an AI assistant should identify itself as such is set out in our article on the labelling obligation under the EU AI Act. And that reliable answers must be usable by everyone is covered in the article on the accessible AI assistant.
How We Build Your Knowledge Base
It starts with an inventory of your content: website, shop, prices, FAQ and documents are reviewed and brought together into a structured knowledge base. Next we define how closely answers are bound to these sources, which areas the assistant answers on its own and from which confidence level it hands over to a human. For businesses with heightened protection needs, this can be combined with European or self-hosted language models and hosting in Germany, as described in the article on data protection and hosting. The result is an assistant that answers from your content and pulls back when unsure.
The entry point is usually a website assistant that answers questions from your knowledge base and captures contacts, and can later be extended with shop, booking and action functions. Because the same knowledge base serves several languages, the assistant answers consistently even when visitors ask in their own language, as the article on the multilingual assistant shows. How a custom-trained assistant fundamentally differs from a kit-based bot is explored in the comparison of a custom assistant versus a standard chatbot. Which package fits your plan is shown in the pricing overview.
Sources and studies