I build chatbots for small businesses. One of the projects was a fitness studio that wanted a bot on its website, in its mobile app and in social networks. The bot answers from the studio's own knowledge base and regulations: prices, trainers, schedule, classes.
The question I get asked most about this project is how it avoids making things up. The short answer is that most of the work went into deciding when the bot should stop answering, not into making it answer more.
Here are the rules we ended up with.
What guests actually ask
If you read the first few hundred messages in a studio chat, the list is short:
- how much a membership costs
- who the trainers are and when they work
- when a class starts
- which direction suits a beginner
- where to sign up if the guest has health restrictions
The first four are easy. A knowledge base covers them. The last one is the one that can go wrong, and I'll come back to it.
Rule 1: answer only from the knowledge base
The bot gets a question, looks for the matching passage in the studio's documents, and answers from that passage. If it cannot point to a source, it does not answer.
This sounds obvious, but it changes how you build the thing. You stop asking "how do I make the model smarter" and start asking "what is in the knowledge base, and who keeps it current". Prices change, trainers leave, the schedule moves. A stale document produces a confident wrong answer, and that is worse than no answer. In this project one person at the studio owns the documents, and the bot's answers are only as good as her updates.
Rule 2: no match means handoff, not a guess
At some point a guest asked whether there is parking near the studio. Nothing in the knowledge base mentioned parking. A model that tries to be helpful will produce something plausible. Ours did not, because there was nothing to answer from, so the conversation went to a person.
Every real conversation has questions like this. You cannot list them in advance, so the rule has to be general: low confidence or no source means the bot hands the dialogue over and says so.
Rule 3: handoff depends on the time
This is a small thing that guests notice. If the guest writes at night and the bot cannot answer, it says that a manager will reply during working hours. If the guest writes during the day, a manager joins the same conversation.
Same trigger, two different behaviours. Without the second one the guest waits for a reply that nobody is going to send until morning, and leaves.
Rule 4: some topics are not the bot's to answer
Guests with health restrictions ask where to sign up. The bot can say which classes exist and what the studio's own materials say about them. It does not tell a person whether they can train, and it does not interpret anything medical. That goes to a person.
The same applies to money: discounts, refunds and exceptions are decided by staff. I keep a short list of such topics in the configuration so that it is easy for the studio to change.
A simplified sketch
This is not the production code, just the shape of the decision:
def reply(question, now): topic = classify(question) if topic in NEVER_ANSWER: # medical, discounts, refunds return handoff(now, reason="restricted topic") passages = search_knowledge_base(question, top_k=3) if not passages or passages[0].score < THRESHOLD: return handoff(now, reason="no source") return answer_from(passages) def handoff(now, reason): log_unanswered(reason) if is_working_hours(now): return connect_manager() return "A manager will answer during working hours." The part I would not skip is log_unanswered. Every question the bot could not answer goes into a list. Once a week someone reads it and decides whether the knowledge base needs a new passage. Parking was on that list.
What I would tell someone starting out
- Write the knowledge base before you pick any tooling. A bot is only as good as its documents.
- Decide the "never answer" topics with the business owner on day one.
- Make the handoff a normal, polite part of the conversation, not an error message.
- Read the unanswered questions regularly. It is the cheapest way to improve the bot.
I'd be curious how you handle the "no answer" case in your own bots, especially outside working hours.
I work on this at ELISEIAVERIN, a studio that builds chatbots, integrations and automation for small businesses. If you read Russian, the page about business chatbot development has more on how we set these up.
