Before launch: which questions your website does not answer for the assistant

The setup assistant reads your website and places a list of gaps beside its proposal; this article shows how to work through it before go-live and where each gap belongs.

In plain language

The setup assistant reads your website. From it, it suggests purpose, role, tone, first sentence and typical questions. Whatever it cannot find on the website goes into a separate list of gaps. The list is a suggestion from the language model, not a check against everything. A gap usually just means: not found in what was read. The detail may be missing or sit on a page that was not read. What visitors look for even without a chat usually belongs on the website. Before launch, test each gap in the trial dialogue.

28 October 2026 13 min read By XICBOT

Key points

The setup assistant reads your website and proposes purpose, role, tone, first sentence and typical questions – from what it read, not from the language model's general knowledge.
Whatever the website does not provide stands as a separate list of gaps beside the proposal: a short list of information without which the assistant could not answer questions with a source, proposed by the language model and not a complete comparison. Where each gap gets closed is up to you.
A gap usually means: not found in what the run evaluated. The information may be missing or may sit on a page that was not read or not evaluated – because of robots.txt, noindex, a foreign address, the limit of 25 pages per run or because the proposal only evaluates a limited excerpt of what was read.
What visitors look for without the chat as well usually belongs on the website. Documents and detailed questions belong in the knowledge base, and some things stay open on purpose and go to a person.
Acceptance happens field by field. The assistant starts as a draft, and before it is switched to active every line of the list of gaps belongs in the test dialogue as a check question.

What the setup assistant proposes

It starts with an address. Instead of facing empty fields, you name your own website, and the run turns it into a proposal. At most 25 pages are read per run, in a fixed order: first the start address, then the sitemap, then the pages already indexed in the previous run, finally the links on the pages read. That text yields five parts of the proposal: purpose, role, tone, first sentence and typical questions. They come from what was read and not from what a language model believes it knows about your industry in general. The page create an account and set up your assistant describes the whole path from account to embedded widget.

Part of the proposal	What it contains	What you check during review
Purpose	What the business offers, in brief	Is the focus right, or has a side page shifted it?
Role	Whom the assistant represents on the website	Does the role fit what the assistant is actually meant to do?
Tone	How it speaks, derived from the language of the website	Does it sound like your business, even when declining?
First sentence	The first message in the opened chat window	Does it say what the assistant answers from, without promising anything?
Typical questions	Questions that visitors to this website are likely to ask	Are these the questions your team really hears? Deselect the ones that do not fit.
List of gaps	What the run did not find for questions that are to be expected	Where will each gap be closed, and who takes care of it?

Every part of the proposal is up for review before anything is created. Purpose, role and every proposed question carry the address of the page they come from. If that origin is not among the addresses that were read, the proposal is dropped before you see it. A fixed word list removes promises and advertising words from the proposal and notes alongside what was taken out. What remains is a draft you work on, not a finished text. What a good first sentence can be measured against is set out in the article on the greeting in the chat widget.

The machine proposes, the human accepts

A proposal is not yet a result. Acceptance happens field by field, the adopted assistant starts as a draft, and the adopted questions then run through the test dialogue like any other question.

The list of gaps: a working list, not an error report

Beside the proposal stands what the website does not provide. This list is not a verdict on the quality of your website and not an error message from the run. It is the working list for the knowledge base: each line describes a question that visitors are likely to ask and that the assistant could not answer with a source. An assistant that may only answer from excerpts of its sources says, for such a question, that nothing is on file and hands over to a person instead of guessing. The list of gaps shows these places before a visitor finds them.

A second list of gaps comes into being later, in operation: if an answer there ends without a single source, it becomes a line, with very short inputs such as a greeting excepted. Both lists share the same goal but rest on a different basis. The list from setup is a forecast drawn from the text of the website; the list in operation is a collection of real questions, as evaluated by conversation analytics. The first helps you close the obvious gaps before launch. The second shows afterwards what nobody thought of. How ongoing maintenance can be organised is described in the article keeping the knowledge base up to date.

Feature	List of gaps from setup	List of gaps in operation
Timing	before launch, from the setup run	after launch, ongoing
Basis	an excerpt of the text read from the website	answers without a single source
What it shows	questions that are to be expected but have no source	questions that were asked and had no source
What it does not show	questions that cannot be foreseen from the text of the website	questions nobody asks because the answer is already elsewhere
Usual remedy	extend the website, add sources	add a source, for frequent questions also the website

What a gap means and what it does not

A line on the list of gaps usually means: in what the run evaluated, there was nothing usable on the matter. That alone does not tell you whether the information is missing from your website. The list itself is a proposal by the language model with a few entries, not a complete comparison between the website and the questions to be expected; whatever is not on it is therefore not automatically covered. Before you close a gap, it is worth looking at the cause, because each cause calls for a different remedy.

The information is missing. Neither the website nor any document says when the emergency service can be reached or which area is covered. That is the actual gap.
The information is on a page that was not read or not evaluated. It sits behind a robots.txt block, under a foreign address or beyond the 25 pages of the run. A page marked noindex is fetched and counted towards the 25 pages, but held back as a draft.
The information is on a page that was read, but not in the excerpt used for the proposal. For the proposal, the language model only evaluates a limited excerpt and relies mainly on the pages read first. The page itself is still in the knowledge base; a check question in the test dialogue shows whether the assistant answers the question with a source.
The information is there, but not as readable text. Opening hours as an image, services in a linked PDF, content that is only loaded via JavaScript: no JavaScript is executed while reading, no text comes out of images, and a PDF only enters the knowledge base as a file of its own.
The information is scattered or vague. Half a sentence in an older post, a footnote in the small print. Whether that holds up as a source is uncertain; a visitor finds it just as hard to locate.

Why pages are missing: robots.txt, noindex, foreign origins

The setup assistant only reads what the website releases to it, and it reads it by the same rules as the knowledge base. The first place where this is decided is robots.txt. The protocol behind it, RFC 9309, specifies where a crawler finds these rules:

The rules MUST be accessible in a file named "/robots.txt" (all lowercase) in the top-level path of the service.

RFC 9309 (Robots Exclusion Protocol), Section 2.3

What matters for the list of gaps is the consequence: an area that robots.txt blocks for the XICBOT crawler is missing from the proposal, and any information that only appears there consequently shows up as a gap. Addresses on a foreign origin, such as an outsourced booking portal, are left aside as well, along with image, media and archive files. A page that sets itself to noindex counts as a draft and gets no sections. Which group of robots.txt applies to the crawler and how your own exclusion patterns for the website source relate to it is described in the article exclusions in the knowledge base.

Cause	How you recognise it	What helps
Block in robots.txt	The path is listed under Disallow in the group for the crawler or, if there is none, in the general group	Check the block; if it is intended, put the information on a released page
noindex on the page	The robots meta element of the page contains noindex	Check the marker on pages that visitors are meant to read; test pages stay on noindex
Foreign origin	The page lives under a different address than the website	State the key information on your own website or add the page as a source of its own
Limit of 25 pages	The page sits deep in the sitemap or can only be reached through many links	Add the page as a source of its own or state the information on a central page
Not in the excerpt	The check question in the test dialogue is answered with a source, although the line is on the list	Treat the line as closed; state important information on a central page as well
No readable text	The information is in an image, a PDF or is loaded via JavaScript	Put the information on the page as text or upload the document as a file

Where a gap gets closed: on the website first

A real gap can be closed in two places: on the website or in the knowledge base. The faster route is often the second, because a free text is written in a few sentences. The better one is frequently the first. What the assistant lacks, the visitor who does not open the chat at all usually lacks too, and so does the search engine that brought them. If information only exists in the knowledge base, website and chat drift apart: the website is silent, and the chat says something the visitor cannot look up anywhere. If it is on the website, it is read with the next fetch of the website source, and both say the same thing.

One piece of information, one place

What visitors look for without the chat as well belongs on the website, and the knowledge base reads it from there. That way each piece of information is maintained in one place, and website and assistant do not contradict each other when it changes.

The knowledge base is the right place for everything that would be too detailed, too technical or too specific for the website: data sheets, care instructions, manuals, detailed answers to specific questions. Besides the website, three further source types are available for this: free text, file and a collection of questions and answers. Some gaps stay open on purpose, such as a question about costs that depend on the individual case. Then declining with a handover to a person is the right answer; how that sounds is described in the article declining instead of guessing. Which content belongs in a knowledge base in the first place is sorted out in the article building a knowledge base.

Entry (example)	Place	Reason
Weekend emergency service	Website	visitors look for it without the chat too, often under time pressure
Area covered for call-outs	Website	decides whether an enquiry makes sense at all
Service interval per unit	Knowledge base, as a file	too detailed for a page, suitable as a data sheet
Bleeding a radiator	Knowledge base, as question and answer	recurring detailed question with a clear answer
Cost of a service visit	left open on purpose	depends on the individual case; the assistant says so and hands over
Contact route for complaints	Website	may touch on mandatory information, see below

Working through the list before go-live

The order saves work because each step makes the next one smaller. A pass in six steps keeps the list manageable, even when several people are involved:

Assign a cause. Each line gets one of the five causes: missing, not read, not in the excerpt, not readable, vague.
Decide on the place. Website, knowledge base or left open on purpose. When in doubt, the website, if visitors need the information without the chat as well.
Settle responsibility. The person who closes a gap on the services page is often not the one who closes a gap in the data sheet. One name and one date per line.
Accept the proposal. The review does not have to wait for the gaps: acceptance happens field by field, and the assistant then stays a draft until you switch it to active.
Close the gaps and have them read in. Changed pages are read with the next fetch of the website source; new files and texts are added as sources of their own. When a source was last read is shown for each source in the knowledge area of your account.
Sign off with check questions. Every line of the list of gaps becomes a question in the test dialogue. Only once it is answered with a source is the gap closed.

The check questions from the list of gaps belong in the fixed acceptance test before go-live, together with the questions your team hears every day. How such a question catalogue is built and which results should be in place before switching to active is described in the article acceptance testing an AI assistant. The wording matters: ask the way a visitor asks, not with the words you have just written on the website. Otherwise the question only tests whether the same words come back.

When a gap concerns mandatory information

Some lines on the list of gaps concern information that a commercial website has to carry anyway. The clearest example is the contact route. The German Digital Services Act requires service providers offering commercial digital services, usually for remuneration, to keep certain information easily recognisable, directly accessible and permanently available, including under Section 5 (1) No. 2 DDG:

information enabling them to be contacted rapidly by electronic means and communicated with directly, including their electronic mail address

German Digital Services Act (DDG), Section 5 (1) No. 2, unofficial translation

For the list of gaps this sets a clear order of priority. Mandatory information belongs on the website, in the imprint for instance; the knowledge base can carry it in addition but cannot replace it. At the same time, the list is not a tool for checking legal duties. It names what the assistant lacks for answers with a source, not what a website must contain by law. A line about the contact route can also mean that the imprint was not evaluated: some operators set the imprint and privacy policy to noindex, and then these pages receive no sections. Whether and which information duties apply in an individual case is a matter for professional advice.

When the run ends empty

A website can yield so little readable text that no proposal makes sense. If less than four hundred characters of readable text remain, the run ends with the finding empty and names the counted figure. That is a result, not an error: an assistant set up from almost nothing would sound plausible and know nothing. The reasons typically lie in the website itself:

The content is only created in the browser via JavaScript; no JavaScript is executed while reading.
The home page consists mainly of images or a video, and the accompanying text is part of the graphic.
robots.txt blocks the crawler entirely or for the areas where the content sits.
The essential information is in linked PDFs; a PDF does not come in through the page fetch but as a file of its own.

The way out is the same as for a gap: change the website so that its information is there as text, or start with other sources. The setup assistant is an additional route, not a mandatory step. Every field can be filled in by hand, and documents, price lists or a question collection carry the assistant even without a website.

What XICBOT takes on during setup

In XICBOT, setup from the website is a run of its own, with at most one open per account. It reads the website by the same rules as the knowledge base: at most 25 pages per run, robots.txt with the crawler's own name, pages set to noindex only as drafts, no foreign origin, no JavaScript. A limited excerpt of what was read, mainly the pages read first, yields purpose, role, tone, first sentence and typical questions. Anything without a source among the addresses read is dropped, and a fixed word list strikes promises and advertising words with a note. Whatever the website does not provide stands beside it as a separate list of gaps.

Reads

Start address, sitemap, pages indexed in the previous run and links, at most 25 pages per run and by the rules of your robots.txt.
Proposes

Purpose, role, tone, first sentence and typical questions, from what was read and up for review before anything is adopted.
Names

Whatever the website does not provide, as a separate list of gaps: the working list for the knowledge base.
Stays with you

Acceptance field by field, closing the gaps and switching to active. The machine proposes, the human accepts.

The assistant starts as a draft and only answers once it has been switched to active and the widget has been released: two separate switches, so nothing ends up on your site by accident. No means of payment is required to sign up. If you want to see which gaps your own website has, create an account and enter the address; the way there is set out under set up your assistant yourself. If you would rather not do the work yourself, you can book the setup as a separate service.

Frequent questions

Do I have to use the setup assistant?

No. It is an additional route, not a mandatory step. Every field can be filled in by hand, and sources can be added without a setup run as well. The run usually saves you mainly the gathering and brings along the list of gaps, which when filling in by hand tends to become visible only once the assistant is in operation.

Why is something on the list of gaps that is on my website?

Usually because the page was not read or not evaluated, or the information is not available as text. Common reasons are a block in robots.txt, a noindex marker, a foreign address, the limit of 25 pages per run or content in images, PDFs or loaded via JavaScript. In addition, the proposal only evaluates a limited excerpt of what was read, and the list is a proposal, not a complete comparison. A check question in the test dialogue shows whether the line is still open before the same information is written a second time.

Do I have to close every gap before the assistant goes live?

Not necessarily. It makes sense to close first the gaps visitors are likely to ask about often. Others can stay open on purpose: the assistant then usually says that nothing is on file and hands over to a person instead of guessing.

Does a gap belong on the website or in the knowledge base?

What visitors look for without the chat as well usually belongs on the website; the assistant reads it from there. Detailed documents such as data sheets or instructions and recurring detailed questions tend to fit better into the knowledge base as a file or as question and answer.

What happens if my website has hardly any text?

If less than four hundred characters of readable text remain, the run ends with the finding empty and names the counted figure. It is then usually worth looking at the website itself, for instance at content that is only created via JavaScript. Alternatively, start with documents, a question collection or fields filled in by hand.

Does the list of gaps replace a legal review of the website?

No. It shows what the assistant lacks for answers with a source, not what a website must contain by law. If a line concerns mandatory information such as the contact route, that information generally belongs on the website. Whether and which duties apply in an individual case is a matter for professional advice.

Ask it yourself, Assistant of this website

Automated assistant

Settings