Setup from your website

Under Setup you enter the address of your website. XICBOT reads its pages and drafts an assistant from them: name, role, tone, opening line, questions with answers and topics it stays silent on. For statements about your business, the proposal names the page they were taken from. None of it takes effect until you accept it with Create assistant.

Where to find setup

Setup is at app.xicbot.de/einrichtung. In the side column it is listed as Setup in the Work group, between Overview and Assistants. The help link of this area points to this page.

At the top of the page, the Setup path shows the four steps Create, Connect knowledge, Test and Embed; setup is the Create step. The steps are links: Create leads to the form for a new assistant under Assistants, Connect knowledge to Knowledge. As long as the account has no assistant, Embed has no link and shows “after step 1”. While “The assistant has been created.” is shown after accepting, Test and Embed lead to the new assistant. Reading the website creates the knowledge source, and accepting the proposal creates the assistant. The next steps are covered in Test and acceptance and Embed the widget.

Who can use setup

With the Read role, the form is headed by the line “With the Read role this area is view-only. Changes need at least the Edit role.” The email address must also be confirmed and the account active. Otherwise the page reports “This area stays locked until the email address is confirmed.” or “This account is locked. Please write to mail@xicbot.de.” Roles are assigned under Team and roles; confirming the address is covered in Sign-in and access.

Enter the website address

The page shows the field Address of your website when there is no run yet or the most recent run has been accepted or dismissed, after Start another run and after Dismiss. Below it you read: “The only basis is your website. What is not there is not in the proposal either.” Below that you choose Deutsch or English under Language of the assistant; the language of the interface is preselected. The sentence below reads “The proposal, greeting and fallback reply are written in this language.”

1. 1

   Step 1

   Enter the address

   Enter the full address, for example https://www.example.com. It has to be reachable from the internet. An address in an internal network is accepted at the start, but the run then ends in leer with the code url_not_allowed.

2. 2

   Step 2

   Start reading

   Choose Read website. The button is disabled while the field is empty; while the run is being started it reads “Loading …”.

3. 3

   Step 3

   Wait

   The page first shows “The website is being read. Depending on its size this takes a few minutes.” and then “The proposal is being written.”, with the address below. It checks the status every eight seconds. You can leave the page: the next time you open it, the run is shown again.

4. 4

   Step 4

   Review the proposal

   Once the proposal is ready, the form appears under Proposal. How to work through it is described further down.

The field checks before sending: it expects an address including the scheme, so the browser will not send an entry such as example.com, and it does not accept more than 255 characters. An address with a scheme other than http or https, such as ftp://example.com, is rejected by the server. The field then shows “Address of your website is not a valid address.” and the top of the page shows “That did not work. Please check your entries.”

What happens while reading

The run works in the background. It creates a source of type website under Knowledge, named after the host name of the address, and reads the website into it. Every run creates a new source of its own, even if one already exists for the same address. The source belongs to your account, not to the run: it stays even if you do not accept the proposal. A separate background job then builds the search index for the source; the proposal does not need it and works from the text that was read. See Knowledge and Background jobs.

• Order: first the address you entered, then the addresses from the sitemap, and finally the links found on pages already read. There is no link depth: reading stops once 25 pages with text have been collected or no address is left waiting.
• Sitemap: taken from the Sitemap lines in robots.txt, otherwise from /sitemap.xml; at most 6 sitemap files of up to 5 MB each, and sitemap indexes are followed. Compressed files are unpacked if the server delivers them as XML, as plain text or without a content type. Once a file brings the count to 100 addresses, no further sitemap is read.
• Same website: only pages with the same scheme, host name and port as the start address are read. If the address you entered redirects from http:// to https:// or to the version with or without www, the target becomes the start address. If a redirect leads to another website, that page is not read.
• Same address: addresses that differ only by a trailing slash or by index.html, index.htm or index.php count as the same page. The part after ? counts.
• Queue: new addresses are added while fewer than 100 are waiting to be read.
• Per page: at most 3 MB, a 20-second timeout and 5 redirects. HTML and plain text are read; links to files such as PDFs, images, videos, stylesheets or scripts are skipped, as are mailto:, tel:, javascript: and data: links. For links containing #, only the part before it counts, so /page#section is /page; a pure anchor such as #top is skipped. At most 2,000,000 characters of a page’s text are stored, and pages with identical text only once.
• Counting: only pages on which text was found count towards the 25 pages. If links remain open at the end that were neither read nor blocked, the source under Knowledge shows: “Only the first 25 pages of this website were read. N links were still open when the crawl stopped.” N counts those links, not the missing pages.

How robots.txt is applied

User-agent: XICBOT-Wissensbegeher
Disallow: /internal/
Allow: /internal/prices.html

The example blocks the /internal/ area for the crawler but still allows the one page inside it, because its pattern is longer. Under Knowledge you can later add Excluded paths to the source; they apply in addition to robots.txt and take precedence over it. A source newly created by setup has none yet.

Pages that are read but not taken in

A page is read but stored without sections if a robots or xicbot-wissensbegeher meta element carries noindex or none, for example <meta name="xicbot-wissensbegeher" content="noindex">. The same applies to a page whose publication date lies in the future: article:published_time is used, otherwise datePublished from a JSON-LD block. A date that cannot be read counts as no date. Such pages are not used for the proposal.

How the proposal is made

The basis is the text of the pages taken in, in the order they were read, up to 40,000 characters in total; anything beyond that is cut off. If fewer than 400 characters come together, no proposal is made. A language model writes the proposal from this text. It is instructed to state only what appears on the supplied pages, to name the page as the source for every statement about the business, to write in the chosen language, German with proper umlauts, and to promise nothing: no prices, deadlines, guarantees, certifications or claims to be the best or the leader. It is asked for a name of at most three words, an opening line of at most 140 characters, five to ten questions, three to eight gaps and at most 8 topics to stay silent on, each of at most 120 characters. These are instructions to the model; what is actually checked afterwards is described in the next section.

What is checked before you see it

A commitment is any entry on a fixed list of words and phrases, for example about guarantees, free services, deadlines, certifications or top rankings. The comparison ignores upper and lower case, and umlauts and ß count the same as their spelled-out forms, so Rückerstattung (refund) and Rueckerstattung are the same word. Most entries also match inside longer words, such as kostenlose (free of charge) or Zufriedenheitsgarantie (satisfaction guarantee). beste (best) only counts at the end of a word, as in bester or allerbeste, not in Bestellung (order) or bestehen; binnen (within) only counts as a whole word, not in Binnenmarkt. For a run in English, an English list applies in addition, for example with guarantee, warranty, free, cheapest, best, within, at the latest, refund, certified and market leader; there, free, best and within only count as whole words. Removed lines leave the note Promise “…” with the entry found; identical notes appear only once. The notes are shown above the form after Removed from the proposal and a colon, separated by semicolons; their wording follows the language of the interface. Older runs show their stored list unchanged. The model’s raw text is not passed on, not even through the API.

Review and accept the proposal

Under the heading Proposal you see the address that was read, the language of the proposal as “Language: German” or “Language: English” and the sentence “Every line can be edited or dropped. Only what you leave standing is accepted.” The proposal is placed in the fields; whatever you change there is the version that counts. What your business does, Who the assistant represents and every question show the Source page as a link to the page on your website. It opens in a new tab so that your review stays in place. If you reload the page or return to it with Back, your changes are back in the fields as long as the run is awaiting your decision. If the checked proposal contains no role, the field Who the assistant represents stays empty and is followed by “No verified source page - not accepted”, even if the model did not propose a role at all. Whatever you type into the field yourself is still accepted.

Under Questions and answers and Topics it stays silent on you read “Ticked lines are included.” Screen readers announce each checkbox with its question or topic and the addition “(include)”, together with whether it is ticked; the answer field takes the question as its name. Create assistant submits your version; the button is disabled while Name of the assistant is empty or more topics are ticked than the never-discuss list holds. The page then reports “The assistant has been created.” and offers Open assistant and Start another run, followed by the gaps of the proposal, see Gaps on your website. The message and buttons stay until you choose a run under Earlier runs or press Start another run. The new assistant is then listed under Assistants.

Gaps on your website

Above the buttons at the end of the form, the section Gaps on your website lists points on which the assistant finds nothing on your website, with the note: “This is the most valuable part: on these points the assistant finds nothing on your website. Add them to your website and then re-read the source under Knowledge, or it will owe your visitors an answer.” Below it, Re-read the website source under Knowledge leads to the Knowledge page. The gaps are not carried over. After accepting, the section stays as Open gaps from the setup, right after creating the assistant and when you open the run later under Earlier runs; while the assistant exists, Maintain facts on the assistant then also leads to it. Add the missing information to your website and then choose Re-read for the source under Knowledge, see Knowledge.

What accepting creates

Create assistant creates an assistant and its first version in a single step. What counts is what you submitted, not what the proposal said. If any part fails, nothing is created. The assistant then appears under Assistants and can be edited there like any other.

In the chat window on your website, the assistant only answers once you set it to active under Assistants. Check its answers in the test dialog first, as described in Test and acceptance.

States of a run

After the colon comes the reason. The server records it as a code, and the page shows it in the language of the interface; the tables below list each code and what is shown. Older runs still carry a German sentence as the reason, which appears unchanged. If a run ends in leer or fehler, start over with Start another run. Dismiss also shows the empty field and in addition keeps the run from appearing again the next time the page is opened.

Messages and their causes

If the website cannot be read

If the website cannot be read, the run ends with “No proposal: The website could not be read.” followed by the message for the code from the following table; a code without a message of its own is followed by “Processing failed.” The same source is listed under Knowledge with the matching message. If reading stops because of an unexpected error, the run instead ends with “The run failed: The website could not be read.” and no further message. The reason refers to the address you entered; a single broken subpage does not stop the run.

If no proposal is produced

Messages when starting and accepting

A plan limit applies as soon as the count has reached it, not only once it is exceeded. Which limits your plan sets and how they are counted is covered in Plans and quotas.

Earlier runs

If there is a run that is not currently shown, the section Earlier runs lists the 20 most recent, newest first, with the address, the start (“started on …”) and the state as a badge: Reading, Drafting, Awaiting decision, Accepted, No proposal, Failed or Dismissed. Screen readers announce the start and state together with the address, and the run on display is marked as current for them. Clicking an address opens that run with its state and proposal. When the page opens, it only shows the most recent run, and only while it has been neither accepted nor dismissed, even if it ended in leer or fehler; otherwise the empty Address of your website field is shown. Start another run returns to that field; the button is offered for leer, fehler and uebernommen, and also below a proposal awaiting your decision. There it is locked while accepting is in progress and for the Read role. Such a proposal is kept and can be opened again under Earlier runs; it does not block a new start. For leer, fehler and below a proposal, the page also offers Dismiss from the Edit role upwards. Afterwards the empty field is shown with “The run has been dismissed. It is still listed under “Earlier runs” and can be opened from there.”, and the run no longer appears by itself for anyone in the team when the page is opened. Nothing is deleted: a dismissed proposal can be opened under Earlier runs and still be accepted. Individual runs cannot be deleted.

What is kept

For each run, the address, language, state, checked proposal, the version you submitted, the note and the start and end times as well as the time it was dismissed are kept. If you delete the knowledge source or the assistant, the run stays and only loses the link to it. Deleting the account deletes all runs as well. The owner’s data export includes the runs. Details are in Retention and deletion and Privacy and legal.

API

The page uses five endpoints. They require a signed-in session, an active account and a confirmed email address. General information on authentication and limits is in API: knowledge and assistants and API limits.

Frequent questions