Skip to content
XICBOT

Settings

Deutsch
AI assistants & automation

Follow-up questions in chat: which next questions the assistant suggests, and which not

Which follow-up questions an assistant should suggest after an answer, why suggestions without coverage are dropped and what analytics shows about clicks.

In plain language

After an answer, the chat can show short questions as buttons. A click then asks just that question.

A button is like a promise. So it should only show questions that have an answer. XICBOT checks the questions first against what the firm knows. Questions with no answer are left out.

The report shows which buttons were there and which ones were clicked. The buttons also work with the keyboard.

15 min read By XICBOT

More on conversation analytics

After an answer, a chat often raises the question of what comes next. An assistant can offer short follow-up questions as buttons for this: one click, and the next question has been asked. That saves typing and shows what the assistant can give information about. It also has a downside. Every suggestion is an invitation, and an invitation to a question the assistant cannot answer ends in a refusal the visitor did not ask for. This article shows which follow-up questions an assistant should offer and which not, how XICBOT checks suggestions against the knowledge base before they are delivered, what conversation analytics shows about suggestions offered and taken up, and what matters when someone operates the buttons without a mouse or with a screen reader.

Follow-up questions: only what can be answered is suggested
Follow-up questions: only what can be answered is suggestedFour candidates from the model, one cap and one coverage check: two questions appear as buttonsSuggestions from the modelCheck before deliveryWhat the visitor seesHow much does shipping cost?covered: shipping costs sectionCan I pay by invoice?covered: question-answer pairIs there a first-order discount?no coverage: ends in abstentionDo you also ship to Switzerland?dropped: cap of three reachedYes, we deliver to Austria.Source: shipping and deliveryHow much does shipping cost?Can I pay by invoice?What is counted afterwardsStoredexactly the list that was shownClickedthe question carries its originEvaluatedoffered and taken up per topicA typed question and the contact offer do not count as a suggestion taken up.
Four candidates from the model, one cap and one coverage check: two questions appear as buttons
Labels in the illustration
  • Suggestions from the model
  • Check before delivery
  • What the visitor sees
  • How much does shipping cost?
  • covered: shipping costs section
  • Can I pay by invoice?
  • covered: question-answer pair
  • Is there a first-order discount?
  • no coverage: ends in abstention
  • Do you also ship to Switzerland?
  • dropped: cap of three reached
  • Yes, we deliver to Austria.
  • Source: shipping and delivery
  • How much does shipping cost?
  • Can I pay by invoice?
  • What is counted afterwards
  • Stored
  • exactly the list that was shown
  • Clicked
  • the question carries its origin
  • Evaluated
  • offered and taken up per topic
  • A typed question and the contact offer do not count as a suggestion taken up.

Follow-up questions: only what can be answered is suggested

What a follow-up question in chat is

A follow-up question is a short question that the assistant places under its answer and that the visitor can take over with one click. It picks up what the answer leaves open or what typically comes next. Someone who has learned that a shop delivers to Austria often asks about shipping costs; someone who knows that a practice is closed on Saturdays asks about the out-of-hours service. The click sends the wording of the button as if the visitor had typed it. The assistant then answers as it would any other question, and suggestions can appear again under the new answer. The result is a conversation that feels its way forward step by step, without the visitor having to phrase every question themselves.

Follow-up questions are easily confused with three related controls that do other jobs. Suggestions before the first question invite the visitor into the conversation and sit next to the greeting; how to word them is described in the article on widget placement and proactive messages. Product suggestions show items, not questions, and belong to guided selling in chat. A contact offer leads to a person and not to another answer from the knowledge base. Follow-up questions, by contrast, start after an answer and lead to the next answer on the same matter. The table sets the four side by side.

ElementWhen it appearsWhat a click does
Suggestion before the first questionnext to the greeting, before the conversation startsasks an opening question
Follow-up questionunder an answerasks the next question on the same matter
Contact offerwhen the answer names a way to a personpasses on the wish to speak to a person
Product suggestionin guided sellingshows an item with details from the catalogue

The benefit lies in two places. First, visitors often do not know what to ask next, or they do not know what the assistant can give information about at all. A good suggestion makes both visible without anyone having to read a list of topics. Second, typing on a phone costs time and space: the keyboard covers part of the window, and a question with a place name or a product name is quickly mistyped. A button with the finished wording saves both. What the narrow window demands otherwise is described in the article on the shop chat on smartphones.

A suggestion is a promise

Whoever clicks a suggestion expects an answer. That is the heart of the matter, and it is stricter than it first seems. If a visitor asks a question on their own and the knowledge base contains nothing on it, a clear refusal is the right response: the assistant says that it has no information on this and names a way forward. Why that is better than a guessed reply is described in the article declining instead of guessing. If the assistant suggests the question itself, however, the case is different. The visitor then gets no answer, and to a question the assistant invited them to ask. That seems contradictory and costs trust, even if the refusal itself is worded correctly.

Suggest what can be answered

A follow-up question is only a good suggestion if the same question, typed instead of clicked, would receive a sourced answer. Everything else does not belong on a button, however obvious it may be in terms of content.

This leads to a check that is easily overlooked. A language model forms follow-up questions from what it has in front of it at that moment: the question, the answer and the excerpts from the knowledge base. Obvious next questions, however, are often precisely the ones the excerpts say nothing about. After an answer about delivery countries, a question about a new-customer discount is close at hand, even if no discount is described anywhere on the website. When the suggested question is clicked later, the assistant searches afresh for it. Whether it can be answered is therefore not decided while it is being worded, but in this new search. A reliable check therefore asks every suggested question in advance the way it would be asked later, and discards it if nothing solid comes back.

How XICBOT selects follow-up questions

In XICBOT, follow-up questions come about in two steps. First, the instruction to the language model asks, at the end of every answer, for at most three questions that the visitor could sensibly ask next and that the available material answers, each one short and without numbering. These lines are markers for the system and do not appear in the answer text. If the model supplies more than three, only the first three count; a question that occurs twice is taken once. Then comes the actual check, before anything reaches the visitor. The list shows the path of a suggestion from the model to the button.

  • Suggest: at the end of the answer, the model names up to three short questions.
  • Limit: anything beyond three is cut off; duplicates are dropped.
  • Check: every question goes through the same search as a typed question. Does it hit a maintained question-answer pair, or do the sections found cover enough of its meaningful words?
  • Sort out: whatever does not reach the threshold is dropped. If no question remains, there is no follow-up button under the answer.
  • Store: the remaining list is saved with the answer, exactly as it was delivered.
  • Display: the widget shows the questions as buttons under the answer.

The threshold is the same one that decides on a refusal under "Check an answer" and in the API: the share of meaningful words in a question that appear in the best section found, or a maintained question-answer pair in the knowledge base that matches the question. For suggestions, it is a hard gate: whatever does not reach it is not offered. In the chat, XICBOT also measures coverage for typed questions and stores it, but does not block the answer there. The stricter yardstick for suggestions is a deliberate choice: the visitor chose a typed question themselves, whereas a suggested one was offered by the assistant, and for that offer the knowledge base has to provide enough beforehand.

Good and weak suggestions

The fact that a question can be answered does not yet make it a good suggestion. A suggestion stands as a single line on a button, with no explanation next to it, and has to work in that format. It should follow on from the answer, that is, hit the next step on the same matter, and not open a new topic that happens to be in the knowledge base as well. It should be worded the way visitors ask, and not like the heading of a subpage. And it should make sense on its own: a button saying "And to there?" makes sense in the conversation, but hardly any later in the analytics, because the wording stands there without the history. The examples show the difference.

Previous answerStrong suggestionWeak suggestionWhy weak
We also deliver to Austria.How much is shipping to Austria?Do you also have a newsletter?opens a new topic instead of the next step
The practice is closed on Saturdays.Who can I reach at the weekend in an emergency?More informationnot a question a visitor would type themselves
The course takes place on Tuesday evenings.Can I do a trial session first?And how does that work?unclear without the history
Returns are free within the return period.How do I register a return?Do I get a voucher when I return something?obvious, but not described in the knowledge base
Delivery usually takes a few working days.Can I track the parcel?Shall I order right away?pushes towards a purchase instead of taking up a visitor question
  • Follows on: the suggestion takes the matter of the last question further.
  • Answerable: the knowledge base contains a sourced answer to exactly this question.
  • Short: one question, one line; what needs two lines is usually two questions.
  • In the visitor's language: worded the way customers ask, with their words rather than in-house terms.
  • Clear on its own: without "there", "that" or "and then?", which only make sense in the history.
  • A real question: no advertising line, no prompt to act and no question from the assistant to the visitor.

Which questions are not suggested

Apart from missing coverage, there are questions that are unsuitable as suggestions even if the knowledge base contains something on them. This mainly concerns matters that require a decision in the individual case, such as goodwill on a late return, and questions whose answer depends on data the assistant does not know in the conversation, such as the status of a particular order. Nor do questions belong on a button that merely repeat what the answer has already said. Anyone checking suggestions should know these groups, because a language model likes to suggest them: they are close in content and sound helpful.

CandidateWhy not as a suggestionWhat helps instead
Do I get a new-customer discount?no discount described in the knowledge base; the click ends in a refusaldropped at the coverage check; if there is an offer, it belongs in the knowledge base
Can you make an exception for me?a decision on the individual case that only a person can makethe contact offer under the answer
Where is my order right now?depends on order data that an answer from the knowledge base does not knowa configured order status lookup or the way into the customer account
Do you deliver to Austria?already in the answerask one step further, for example about the costs
a fourth and fifth questionthe row turns into a second answerat most three, the most obvious first

For the second of these cases, XICBOT has a path of its own. If an answer names a way to a person for the visitor, such as telephone, e-mail, form or appointment, the model sets a separate marker for it, and the widget shows a button "Talk to a person" in the same row. This button is not a follow-up suggestion. It does not go through the coverage check, because it promises not an answer from the knowledge base but a contact, and a click on it does not count as a follow-up question taken up in the analytics. That keeps the number of suggestions taken up clean and stops it from mixing with the wish for a conversation.

Typed or clicked: what conversation analytics shows

Whether suggestions help can only be seen in operation. For that, the analytics must keep two things apart: what was offered and which of it someone clicked. XICBOT stores, with every answer, the list of suggestions that were actually delivered, that is, after the coverage check. Every visitor question records whether it came from an offered suggestion. The widget reports this origin itself, because only there is it known whether a text was typed or clicked. A typed question does not carry the origin, even if its wording happens to match a suggestion, and neither does the contact button.

Conversation analytics brings these details together in the topic list. For the selected period, the top of the list shows how many follow-up questions were suggested and how many of them were taken up; each topic shows the suggestions offered and taken next to the conversations concerned and the share of handovers. Topics are formed from the questions asked, not from the offers: a suggestion nobody clicks does not create a topic of its own but is assigned to an existing topic if it fits. No offer is counted twice. Conversation analytics is included in the Pro and Custom packages.

ObservationPossible readingWhat you check
Many suggestions offered, hardly any taken upThe suggestions do not hit what visitors want to know nextthe conversations of the topic: which question is typed instead?
A topic with many typed questions but no offerNo covered suggestions arise for this matterwhether the knowledge base describes the topic the way visitors ask about it
Many suggestions taken up, followed by handoversThe answer to the follow-up question is not sufficientthe answer to the follow-up question and the source behind it
Suggestions taken up with poor ratingsThe question fits, the answer does notthe section the answer comes from
Few offers overallMany answers receive no covered suggestionwhether the knowledge base contains the obvious next questions

Restraint helps when reading these figures. A suggestion taken up shows that the question interested someone, not that the answer to it helped. How the answer landed is shown rather by visitor ratings and the further course of the conversation; how to gather and read this feedback is described in the article on ratings in chat. Conversely, a suggestion that is rarely clicked is not necessarily bad: some visitors simply have everything they need after the first answer. What is most telling are changes when comparing two periods of equal length after you have changed something in one place.

Accessibility: suggestions without a mouse and with a screen reader

Follow-up questions are controls, and what applies to every control in the chat applies to them: they must be reachable and operable with the keyboard, have a visible focus and carry a name that a screen reader can read out. The simplest way to achieve this is with ordinary buttons whose label is the wording of the question. In XICBOT, follow-up questions are implemented this way: as buttons that carry only text and contain no nested clickable element. The widget can be operated with the keyboard, has a visible focus and works with screen readers; what this means in detail is answered under the question whether the chat can be operated accessibly.

The moment the suggestions appear is harder. Someone working with a screen reader does not see that new buttons have appeared under the answer. The focus should not jump at that point, because a focus that suddenly moves from the input field to a suggestion interrupts typing. For notices without a change of focus, WCAG 2.2 provides status messages: under Success Criterion 4.1.3 (WCAG 2.2) they are to be programmatically determinable through role or properties, so that assistive technology can present them without them receiving focus. The wording is in the following quotation.

In content implemented using markup languages, status messages can be programmatically determined through role or properties such that they can be presented to the user by assistive technologies without receiving focus.
W3C, Web Content Accessibility Guidelines (WCAG) 2.2, Success Criterion 4.1.3 Status Messages, Level AA

Whether newly appearing suggestions are a status message in this sense is a matter of interpretation. WCAG 2.2 understands it as a change in content that is not a change of context and that informs the user about the success or results of an action, a waiting state, the progress of a process or errors. The answer itself is most likely such a result; whether the buttons beneath it belong to it can be seen differently. A short notice such as "Suggestions available" in an area that screen readers announce without a change of focus can help users find the buttons. Whether and which accessibility requirements apply to your offering by law, for example under the BFSG, is a question for the individual case; the article on the accessible assistant gives an overview.

A few rules for designing the suggestion row can be derived from this, and they apply regardless of the system in use. They concern the focus, the label, the size of the buttons and the moment a row disappears again.

  • The focus should stay in the input field when suggestions appear; anyone who wants to use them reaches them via the keyboard.
  • The button label should be the full wording of the question, without a symbol and without abbreviations.
  • For pointer inputs, WCAG 2.2 requires a target size of at least 24 by 24 CSS pixels (WCAG 2.2, Success Criterion 2.5.8), with exceptions; on a phone, larger buttons with space between them help.
  • Outdated suggestions should disappear as soon as a new question has been sent, so that no row belonging to an earlier answer is left standing.
  • In the reading order, the suggestions should come after the answer they relate to.

Checking suggestions before launch

Whether an assistant's suggestions are any good can be checked before launch with little effort. The basis is the questions your customers really ask, for example from the inbox or the telephone. Ask each of them in the widget on a test page of your website where the assistant is embedded, and look at what is offered under the answer. Follow-up questions do not appear in the trial dialogue or in the test conversation of the app, only in the widget. Then click every suggestion. It should lead to an answer with a source that fits the matter. With the XICBOT coverage check, suggestions without coverage have already been sorted out; whether the answer holds up in substance, however, only reading shows.

  • For each typical question in the widget: are suggestions offered, and do they hit the next step?
  • For each suggestion: does the click lead to an answer with a source that really answers the question?
  • If an obvious suggestion is missing, such as shipping costs after an answer about delivery countries, the topic is probably not in the knowledge base, or not in visitors' words.
  • If a suggestion reads like advertising or repeats the answer, the assistant's instruction needs sharpening.
  • After every change, ask the same test questions again in the same wording and compare the suggestions.

Missing suggestions are often the more revealing finding. If not a single follow-up question appears for a topic visitors ask about regularly, the cause usually lies not in the suggestion logic but in the knowledge base: the topic is missing, out of date, or only in a document that does not contain the words visitors use. How to maintain content so that it stays current and findable is described in the article on keeping the knowledge base up to date.

What XICBOT provides for this

In XICBOT, the building blocks for follow-up questions interlock. The instruction to the model asks for at most three short questions that the available material answers. Where a knowledge base exists, every suggestion is checked against its coverage threshold before delivery, and whatever fails the check does not appear. Answers come from your knowledge base and name their sources; if the material is not sufficient, the assistant says so instead of guessing. The widget shows the suggestions as buttons with plain text; a click asks exactly that question and reports its origin. Conversation analytics records per topic what was offered and what was taken up. The page with the features of XICBOT gives an overview of all building blocks. During setup, we test typical questions from your customers and check the answers against the sources.

Frequent questions

How many follow-up questions should an assistant offer?

As a rule, a few are enough. XICBOT shows at most three per answer, and often fewer, because suggestions without coverage in the knowledge base are dropped beforehand. More buttons usually do not make the row more helpful; beyond a certain number it reads like a second answer.

Why do some answers have no suggestions?

Usually none of the model's suggestions passed the coverage check, or the model saw no fitting next question. Often the obvious next questions are not described in the knowledge base, or not in visitors' words. If suggestions are missing for a frequent topic, it is worth looking at the knowledge base.

Can a clicked suggestion still lead to a refusal?

Yes, that cannot be ruled out. The check measures whether the knowledge base contains enough on the question; whether the model turns that into a sourced answer is only decided on the click. In addition, the knowledge base can change between the offer and the click. In conversation analytics, such cases can show up as a knowledge gap or an unanswered question.

Does a typed question with the same wording count as a suggestion taken up?

Not according to the way XICBOT counts. A question counts as taken up if it was sent via the button; the widget reports this origin itself. A typed question with the same wording counts as typed, and the button to contact a person does not count as a follow-up question either.

Do newly appearing suggestions have to be announced to screen readers?

That is a matter of interpretation. Success Criterion 4.1.3 of WCAG 2.2 covers status messages that assistive technology can present without a change of focus; whether new buttons fall under it can be seen differently. A short notice in an area that screen readers announce can help. Which requirements apply to your offering needs to be checked in the individual case.

Can I specify which follow-up questions are offered?

Not as a fixed list per answer. The suggestions arise from the material found for the question and go through the coverage check. You exert influence mainly through the knowledge base: what is described there in visitors' words can be suggested. Suggestions before the first question, where a chat offers them, belong to the greeting and follow their own rules.

What do your visitors actually ask?

Conversation analytics counts topics, unanswered questions, knowledge gaps, drop-offs and handovers from your own conversations.

Your enquiry has arrived. We will get back to you personally.

Enquiry in three steps

  1. 1. Topic
  2. 2. Your case
  3. 3. Contact

What is this about?

One click is enough, everything else is optional.

Tell us briefly about your case

Everything in this step is optional. It helps us reply with something concrete straight away instead of asking back.

Enquiries per month, roughly (optional)
When should it start? (optional)
How can we reach you?

Name and email address are enough. We reply personally.

We use your details to handle your enquiry. More on this in the privacy policy.