Rate limiting in chat: requests per visitor

How to justify a cap per visitor, how it differs from a cap per tenant, and why a waiting notice works better than an error message in the chat window.

In plain language

A chat needs a limit for requests. A limit per visitor slows down a single person who sends too much. A limit per account keeps cost and load in check. The two belong together. The number follows the pace of a person: read, think, type. When the limit is reached, the chat shows a note to wait instead of an error. Many visitors with few questions are not an attack. The system should react to that in a different way.

September 8, 2026 14 min read By XICBOT

Key points

A cap per visitor fends off sustained fire from a single person. A cap per tenant keeps the cost and the load of the whole account within a frame. Both together, not one alone.
The number derives from human pace: how long does somebody need to read an answer, think for a moment and type the next question?
In our own configuration the cap per visitor on the public widget interface stands at 12 requests per minute; arithmetically that allows one question every five seconds.
On hitting the cap, a waiting notice belongs in the conversation, not an error message. The visitor has to learn that nothing is broken and when they can carry on.
A genuine load peak looks different from an attack: many visitors with few questions versus few visitors with many questions. The response must therefore not be the same.
Availability and resilience are not an extra: Article 32 (1) (b) of the General Data Protection Regulation names them explicitly as a protection goal.

What a cap actually helps against

A chat assistant on a public page has an interface anybody can reach who opens that page. This is not an oversight but the precondition for a visitor being able to ask a question without signing in. The flip side: that interface cannot tell by itself whether the other end is a prospect wanting to know whether a delivery will arrive before the weekend, or a program having text generated once a second. Both send a valid question to the same address. What separates them is not the content but the pace and the volume.

This is exactly where rate limiting comes in. It does not judge what somebody asks - that is the job of the system instruction and the knowledge base - only how often they ask. That makes it the only protective layer that takes effect before a request causes any cost at all. Everything that checks content has to accept the request first, process it and then discard it. The cap turns it away while it has cost nothing yet. It replaces none of the other layers, but it is the cheapest of them.

Abuse of the interface

Anyone using somebody else's assistant as a text tool for their own purposes is not asking about the offering but submitting tasks. Without counting, that carries on unnoticed, often for weeks.
Cost nobody ordered

Every request consumes compute time, regardless of whether it comes from a customer. The cap is the only place where that consumption can be bounded from above before it arises.
Availability for everybody else

A visitor who saturates an interface on their own takes it away from everybody else. So the cap works above all for those who never get to see it themselves.

Justify the number, do not guess it

A cap without a justification is a number somebody once entered and which nobody afterwards wants to touch, because nobody knows what happens if it changes. That is exactly why it pays to write the derivation down once - it fits in three sentences and turns a setting into a decision you can talk about later.

The starting point is human pace. A visitor sends a question, waits for the answer, reads it and phrases the next one. The answer alone costs time: in our own measurement series the time to the first character of the answer was 1.3 s at the median, with a coverage of 64 questions to our own assistant, not to customer projects. On top of that comes the time until the answer stands complete, and the time to read it. Anybody staying in that loop will barely get below several seconds per round, even with short answers and fast typing. A cap set clearly above that lets every genuine conversation through and only catches what is no longer a conversation.

On that basis, our own configuration sets a cap per visitor on the public widget interface of 12 requests per minute. It applies per visitor and assistant; arithmetically that allows one question every five seconds. The number is therefore roughly twice what a briskly working person reaches in conversation - headroom for somebody who cuts an answer short and immediately asks again, but too little for a script that reads nothing. That this limit is real and not merely on paper shows in our own measurement run: it waits 5.5 seconds between two questions and thereby deliberately keeps clear of our own brake.

Pace of the conversation: how long does one round of question, answer, reading and next question take? The cap belongs above that value, with room to spare.
Behaviour on cancellation: if somebody cuts a long answer short and immediately asks again, several requests arise in quick succession. The cap has to tolerate that.
Length of the window: one minute is a sound compromise. A one-second window catches every bout of impatience; a one-hour window notices an attack only once it is over.
What happens on hitting it: a waiting time is something other than a refusal. Fixing only the number and not the response leaves half the decision open.
Who may change the cap: a number that can be adjusted in passing is not a protective measure. The change belongs in the same place as any other configuration, with a date and a reason.

A cap a customer reaches is too low

The test of a rate limit is not how many attackers it turned away but how many customers it hit. If a genuine conversation trips it regularly, the visitor is not too fast - the number is too small. The other way round: a limit that does not bite once in months is either set very high or it is not counting what it is meant to count.

Two caps: per visitor and per tenant

A cap per visitor slows exactly one. That is intended and enough in most cases. It does not help, though, where many different identifiers ask at the same time - whether because a campaign runs unexpectedly well, or because somebody deliberately spreads requests across many sessions. Each individual one then stays neatly below the limit, and in aggregate a load still arises that nobody planned for. That is why a second cap is needed at a different level: per tenant, meaning per account and assistant.

Level	Cap per visitor	Cap per tenant
Defends against	sustained fire from one source	the sum of many sources
Counting unit	per session and assistant	across all visitors of one assistant
Typical window	one minute	hour, day or billing period
Who sees it first	the individual visitor	the operator first, then every visitor
Correct response	waiting notice in the conversation	message to the operator, decision on raising it
If it is missing	one person ties up the interface	a wave feeds straight through into cost

The difference is not merely technical, it is organisational too. The cap per visitor is a decision about usability: it must not disturb the customer. The cap per tenant is a decision about money and operations: it must not sit so high that an incident only surfaces in the next invoice, and not so low that a good day in the peak season ends as an outage. Anyone planning for request peaks in the season plans both numbers with it.

What the visitor sees when the cap bites

A rate limit is at heart a refusal: the request does not get answered. On the technical level that is a status code; on the human level it is a moment of uncertainty. The visitor does not know whether they did something wrong, whether the page is broken or whether their question was unwelcome. The wording decides which of the three readings they take away.

The worst variant is the red box with an error number. It tells the visitor something is broken, and that is not true: everything works, they are just too fast. The second-worst variant is nothing at all - the input stays there, the answer never comes, and the visitor tries three more times, which drives the counter up further. What remains is a calm line inside the conversation itself, in the same place the answer would otherwise stand.

It stands inside the conversation, not in a window above it. The visitor should keep seeing the history.
It says that nothing is broken. One sentence is enough: too many questions in a short time just now.
It names a waiting time, and a concrete one. “This will continue in a few seconds” is usable, “please try again later” is not.
It shows another route: the contact form, the phone number or the handover to a person.
It does not repeat itself on every further attempt but replaces the previous line. Four identical warnings stacked up read like a fault.
It does not discard the typed question. Somebody who has to retype their text is more likely to leave than to wait.

The difference between “not now” and “not with you”

A rate limit always means “not now”. Every phrasing that instead sounds like “not with you” - suspicion, block, violation - as a rule hits a customer rather than the party it was meant for. The attacker does not read the message anyway.

A queue instead of an error message

Between “answer arrives immediately” and “request rejected” lies a third state that is almost always the better choice in conversation: the request is accepted and waits. Technically that means a request above the cap is not discarded but placed in a short queue which empties within a few seconds. To the visitor it looks like an answer taking a little longer - an experience they know and which does not alarm them.

The reason this works lies in the orders of magnitude. In our own measurement series the time to the first character of the answer was 1.3 s at the median, measured across 64 questions to our own assistant. A visitor who occasionally waits four seconds instead of 1.3 experiences a slower answer, not a malfunction. A visitor who gets an error message instead experiences an outage - even though they objectively waited less long. Anybody who does not know the times of their own assistant cannot make this trade-off; the route to them is described in measuring response time instead of guessing.

The queue needs an upper bound

A queue without a bound is no protection, only a postponed collapse. Beyond a set length requests are refused again - but then with the notice described above.
The waiting time needs an upper bound

Anybody waiting longer than a usual answer takes notices the difference. From roughly twice the habitual time onwards, a visible cue beats further silent waiting.
The visitor needs a sign of life

A moving dot or a line reading “your question is queued” costs nothing and turns an empty area into a recognisable state.

A genuine load peak or an attack

Both produce the same curve on the chart: the number of requests goes up. Yet they call for opposite responses. To a genuine peak you respond by creating headroom - higher limits, more capacity, if need be another person in support. To an attack you respond by closing down. Confuse the two and you throttle on the day of your best demand, or you pay for a wave that brings in nothing.

Feature	Genuine load peak	Abuse
Distribution	many identifiers, few questions each	few identifiers, many questions each
Course over time	rises and falls with site traffic	stands independent of it, often as high at night
Question pattern	spreads across the whole offering	repeats itself or does not fit the subject
Origin	follows the usual spread of visitors	clustered in few networks or with no referring page
Correct answer	raise limits, add capacity	hold limits, refuse the identifier, record the incident

For this distinction to be possible at all, the rate limiter has to record whom it slowed and when. A counter that only knows one number cannot say the day after whether it was spread across thirty visitors or concentrated on one. What belongs in the record is at least the timestamp, the identifier in shortened form, the assistant affected and whether the cap per visitor or the cap per tenant bit. That answers the question in a few minutes rather than leaving it to be argued about. What the handover to a person looks like in this situation is described in handover to human agents.

Resilience is an obligation, not an extra

Rate limiting tends to be treated as an operational detail you set up if time allows. Data protection law takes a different view: the availability and resilience of systems stand there explicitly among the protection goals for which technical and organisational measures are to be taken. A chat assistant processes personal data as soon as somebody asks a question containing a name, an order number or an address - so the provision applies to it too.

the ability to ensure the ongoing confidentiality, integrity, availability and resilience of processing systems and services;

Regulation (EU) 2016/679 (GDPR), Article 32 (1) (b)

The sentence names no procedure and no figure - it demands a capability. Rate limiting is one of the few measures with which that capability can be evidenced without having to describe it: it sits in the configuration, it has a value, and it leaves traces when it bit. For documentation under Article 32 that is worth considerably more than a declaration of intent. Where the processing itself is governed and who carries it out on instruction belongs in the same folder; the allocation is described in data protection and hosting.

Where a blanket cap gets in the way

A single number for every case is convenient and in some situations wrong. Not every request to an assistant is a question from a visitor, and not every visitor behaves like the average. Anyone who fixes the cap once and never looks at it again gets the bill precisely where the assistant was supposed to be earning money.

The logged-in area: a signed-in customer is known, their behaviour is attributable, and they often ask about orders in rapid succession. A separate, higher cap belongs there.
Internal use: an assistant that staff work with runs differently from one on the home page. Rapid follow-up questions are the normal case here, not the exception.
Connected tools: if the assistant fetches data itself while answering, several operations arise from a single question. Put both in the same counter and you throttle yourself.
Accessible operation: anybody working through voice input or assistive technology corrects more often and therefore sends more often. A tight cap hits first exactly those who deserve it least.
The acceptance phase: a test run before go-live deliberately sends many questions in a short time. Without a separate access path for it you end up measuring your own brake instead of the assistant.

What belongs in the report

A rate limit nobody looks at is a setting without feedback. It can have been slowing every second customer for months without anybody noticing, because those who were slowed rarely complain - they leave. So a handful of figures belong in the same report as the other numbers about the assistant, and with a date on them.

How often the cap bit

The plain count per week, separated by cap per visitor and cap per tenant. A jump in it is a reason to look, not a reason to worry.
Across how many visitors it was spread

The same count across three identifiers is a different event from the same count across three hundred. Without that split, abuse cannot be told apart from demand.
What happened afterwards

Did the visitor carry on asking, use the form or close the window? That single column says more about the level of the cap than any discussion about it.

From those three figures the decision follows on its own. If the cap bites rarely and across few identifiers that then disappear, it is working as intended. If it bites often and across many visitors who subsequently close the window, it is too low and it is costing enquiries. If it never bites, it is either set generously or it is counting in the wrong place - you want to know either way before somebody else finds out. Which other reports belong to operations is described under conversation analytics. What a measurement looks like that discloses such limits instead of hiding them is shown in our measurements.

Frequent questions

How many requests per minute make sense?

There is no universally valid figure, because answer length, the cut of the questions and the user group differ from project to project. What can be stated is our own setting: in our own configuration the cap per visitor on the public widget interface stands at 12 requests per minute, per visitor and assistant; arithmetically that allows one question every five seconds. The derivation matters more than the number: measure how long one round of question, answer and reading takes in your project, and set the cap clearly above it.

What happens when a genuine customer reaches the cap?

They get a calm line inside the conversation, not an error message: that many questions arrived in a short time just now, that it will continue in a few seconds, and where else they can go - contact form, phone or handover to a person. The typed question stays in the input field. More important than the wording, though, is the reporting behind it: if genuine customers see this notice regularly, the text is not the problem, the number is.

Why is a cap per visitor not enough?

Because it only sees one source. If requests are spread across many sessions, each individual one stays neatly below the limit and in aggregate a load still arises that nobody planned for. The same effect occurs with no ill intent at all when a campaign runs unexpectedly well. That is why a second cap sits alongside it, per tenant - that is, per account and assistant - over a longer window. It does not protect the individual visitor but the account: against cost and against an overload that would then hit every visitor at once.

How do I tell a genuine peak from abuse?

By the distribution, not by the height. A genuine demand peak consists of many identifiers with few questions each, it follows the rest of the site traffic, and the questions spread across the whole offering. Abuse consists of few identifiers with many questions each, it stands independent of the time of day, and the questions repeat or do not fit the subject of the site. This distinction presupposes that the rate limiter records whom it slowed and when - a bare total counter cannot answer it.

Does a queue make answers slower?

For requests that sit below the cap anyway, nothing changes - they pass through as before. The only thing that gets slower is what would have been refused without a queue, and for the visitor an answer after four seconds is a markedly better experience than an error message after half a second. Two bounds belong to it, though: an upper bound on the length of the queue and one on the waiting time. Without them the queue is no protection, only a postponed collapse.

Is rate limiting a measure under the General Data Protection Regulation?

It can be classed as one. Article 32 (1) (b) of the General Data Protection Regulation names, among the appropriate technical and organisational measures, expressly “the ability to ensure the ongoing confidentiality, integrity, availability and resilience of processing systems and services”. Rate limiting contributes to availability and resilience, and it has the advantage of being evidenceable: it has a value in the configuration and leaves traces when it bit. It therefore replaces none of the other measures but supplements them at a point that otherwise often stays unevidenced.

Ask it yourself, Assistant of this website

Automated assistant

Settings