Adding a chat interface to a website is easy. Making it answer company-specific questions reliably is the real engineering and content problem.
A support assistant needs more than a capable language model. It needs a controlled set of sources, a retrieval workflow, a testing process, and a clear fallback when the available knowledge is insufficient.
This article describes a practical website-to-answer workflow for small and mid-sized products. It is based on lessons from building Oscar Chat, but the principles apply to any retrieval-based customer-support assistant.
Start with source ownership
Before crawling anything, define which pages the business considers authoritative.
A public website often contains several versions of the same information. There may be an old campaign landing page, a current pricing page, archived documentation, translated pages, and blog posts that describe features no longer available.
The crawler can collect all of them, but collection is not the same as truth.
A useful source policy answers three questions: which paths are allowed, which paths should be excluded, and which page wins when two sources conflict.
For a first implementation, stable pages are the safest source set: documentation, product pages, FAQs, shipping and return policies, service descriptions, and current pricing explanations.
Separate extraction from indexing
A crawler retrieves pages. A knowledge pipeline turns those pages into usable evidence.
The extraction layer should remove repeated navigation, cookie banners, unrelated footer content, and other text that adds noise. It should preserve page titles, headings, meaningful paragraph boundaries, and the source URL.
The indexing layer can then divide the cleaned content into retrievable units. The exact chunking strategy depends on the page structure, but a useful chunk should retain enough context to make sense when retrieved independently.
A sentence such as “It is available for 30 days” is not useful without the heading or preceding description that explains what “it” means.
Metadata is equally important. At minimum, keep the page URL, page title, section heading, language, and crawl time. These fields help with debugging, filtering, and future refreshes.
Retrieval needs an uncertainty policy
A common failure mode is allowing the assistant to answer whenever it can generate plausible text.
For customer support, plausible is not the standard. The answer should be supported by the retrieved source.
The system needs an uncertainty policy: if the available evidence is weak, conflicting, or missing, the assistant should say that it cannot confirm the answer and offer a human handoff or contact form.
This behavior is especially important for pricing, refunds, legal terms, medical information, financial services, and order-specific questions.
The best fallback is not a generic error. It should tell the visitor what can happen next: leave contact details, start live chat, open WhatsApp, or send the question to the appropriate team.
Build an evaluation set from real conversations
Testing only a few ideal questions creates false confidence.
A practical evaluation set should include direct questions, informal questions, spelling mistakes, questions that combine two topics, unsupported questions, and questions where the correct response is a human handoff.
For example, a delivery policy might be tested with “How long does shipping take?”, “Will this arrive before Friday?”, “delivery netherlands?”, and “Can you guarantee arrival tomorrow?” These questions may refer to the same source but require different levels of certainty.
Review answers for factual accuracy, source relevance, completeness, tone, and appropriate refusal. Save failed questions and rerun them after every meaningful change to the content or retrieval workflow.
Treat refresh as part of the feature
Website knowledge becomes stale.
Prices change. Policies change. New pages are added and old pages remain accessible. A production workflow therefore needs a refresh strategy, even if the first version relies on manual retraining.
Track the crawl time and source URL. Detect deleted pages. Make it possible to remove an obsolete source. If the website is large, prioritize important sections rather than recrawling everything without control.
The operational question is simple: when a customer-facing page changes, how quickly should the assistant learn the new information?
A simple product workflow
For Oscar Chat, we reduced the user-facing process to four steps: open the Train section, add the website URL, wait while the content is collected, and test the result in the Playground.
The simple interface sits on top of a more important product principle: users should be able to start with existing content, inspect what the system learned, and improve the knowledge with files or approved answers.
That is more practical than asking every small business to design a retrieval system or manually recreate its website as a knowledge base.
Final takeaway
The quality of an AI support assistant is constrained by the quality, structure, and freshness of its sources.
Model selection matters, but it cannot repair contradictory policies or supply facts that the business never documented. A reliable implementation therefore treats crawling, content cleanup, source control, evaluation, and human fallback as first-class parts of the product.
Disclosure: I am the founder of Oscar Chat, the product used as the practical example in this article. More information is available at https://www.oscarchat.ai/ai-chatbot/.
Top comments (0)