How to fine-tune your AI product advisor

1Collect the answers that need work

Real questions from your customers show what to improve. Open Chatbot → Chat History: it lists the conversations from your store, newest first. Search with Search messages with any keyword for the topics that matter to you, for example a product line, “delivery” or “return”.

Let customers tell you which answers did not help. In Chatbot → Chat Settings & Embed, tick Enable message flagging in the appearance settings and save. A flag button then appears next to each AI answer in the chat window; translate its Flag button tooltip into the language of your chat. In Chat History, Show only flagged sessions lists the conversations with an answer marked Flagged by user. A conversation the customer reported is marked Reported.

Two more sources:

• Tickets: the conversations the chatbot handed over to your team show what it could not finish.
• Your own team: the questions customers still send by e-mail or ask by phone.

Chat History keeps a conversation for 30 days after its last message. Select Export to CSV if you want to keep a record. From all of this, write down 20–30 real questions together with the answer you expect. This list is your test set for step 6.

2Find out why the chatbot answered that way

Under each answer of the chatbot in Chat History and in Test Chatbot, select Analyze why AI answered like this. The analysis reads the conversation, the chatbot settings and the knowledge-base content the chatbot used, and it searches your knowledge base and product catalogue again. It only reads: it changes nothing in your data or settings.

The result names the main cause with a confidence (High, Medium or Low), lists the Evidence, and gives a Recommended fix with Who should handle this: Your team or Chaterimo support.

The analysis is automated and can be wrong, and the app says so under every result. Use it as a starting point and check the evidence.

The analysis runs on your own API key, or on Chaterimo's keys if your plan uses Chaterimo API keys. The number of analyses per hour is limited. If you analyse the same answer again shortly afterwards, the app shows the earlier result (Loaded previous analysis).

3Fix the knowledge base

The knowledge base is everything the chatbot searches before it answers: products, web pages, files and texts. Fix it first, because neither instructions nor a better model can correct an answer that is missing from your data.

Add what customers ask about. Typical gaps are shipping and payment, returns and complaints, warranty, contact details and opening hours, size charts, care instructions and compatibility. Open Knowledge Base → Get Data and add each topic as its own entry: paste text in Text Input (each save is a separate entry), import a page in Website Scraping → Scrape a single link, or upload a document in File Upload. Give each entry a clear name, so you can find and update it later. Write tables as one line per row, with the unit or currency in each line.

Remove what misleads. Old promotions, old shipping prices, duplicate pages and pages without useful text (cart, login, search results) make answers worse. Open Knowledge Base → Knowledge Base, filter by type (Text data, XML feeds, Files, Scraped URLs, Products), open the entry and select Delete. For a sitemap that is imported every day, open Filters and list the paths under Do not import these pages from the sitemap., for example /cart/*. Product pages (/products/*) are excluded by default when your products come from a store integration or a product feed. When you scrape pages, enter the classes of menus and footers under Advanced → HTML classes to ignore, so their text does not end up in every entry.

Improve your product data. The chatbot recommends products by what your product data says. From a product feed (XML), Chaterimo reads the title, description, brand, price, product type, availability, image link, product details (g:product_detail) and recommended accessories (recommended_accessories). Put every parameter customers ask about, such as size, material, power or compatibility, into the description or the product details; a parameter that appears only in a product image is invisible to the chatbot. Then:

• Keep prices and availability current: turn on Download this feed again every day and retrain for the feed. Store integrations (Shopify, Shoptet, Upgates, Eshop-rychle) update your products by themselves, so improve the texts in your store administration.
• Set Select Language of your XML feed to the language of the feed.
• If you run several language versions or stores, open each feed in the Knowledge Base and choose under Which chat widgets can use this feed? which chat widget may use it. A feed attached to the wrong chat widget is what the analysis calls a configuration problem.
• Add products that are not in any feed in Get Data → Individual Products.

Explain your order statuses. If customers ask about their orders, open Get Data → Order Details and explain what each order status means and how long shipping usually takes. The chatbot uses this text only when a customer asks about an order by its number. The order status lookup itself needs a connected store (Shopify, Shoptet, Upgates, PrestaShop or Eshop-rychle) and Enable order fetching via API in Chat Settings & Embed (a paid feature).

Keep pages current. On the entry of a scraped page that changes often, tick Scrape this page again every day and retrain. Sitemaps you subscribed to are checked every day at 3:00 AM for new, changed and removed pages.

4Write clear instructions in the chatbot role

The chatbot role is the set of instructions the model gets with every question (its system prompt). It says who the chatbot is and how it behaves, not what it knows. Edit it in Chatbot → Chat Settings & Embed → Chatbot role, or in Knowledge Base → Get Data → Chatbot Role (Instructions), where the app also explains how to write it. Copy as Markdown there copies a brief you can paste into ChatGPT, Claude, Gemini or Grok to draft the role with their help.

The app recommends 15–40 lines in five parts:

1. Who you are: the chatbot's personality in two or three sentences.
2. How you communicate: tone, language and length of answers.
3. Rules: conditional rules such as “share X only when Y”.
4. Escalation: when to offer to pass the customer to a person.
5. Examples: two or three ideal questions and answers.

A short role in that shape can look like this (replace the parts in brackets):

```text You are the product advisor of [store name]. You are friendly, practical and save the customer's time. Answer in two or three short sentences; give details only when the customer asks for them. Recommend at most three products and say in one sentence why each one fits. Share our contact details only if you cannot solve the question. If the customer is upset, acknowledge it first, then solve the problem. If the knowledge base does not answer the question, offer to pass it to our team.

Example: Customer: [a real question from your Chat History] You: [the answer your best salesperson would give] ```

Say what the chatbot should do rather than listing what it must not do; the app notes that long lists of prohibitions make a chatbot hesitant. Know the limits of the role:

• Instructions cannot add knowledge. Prices, products and policies belong in the knowledge base, where they are searched and kept up to date. A price written into the role is soon outdated.
• Longer is not better. A long role makes the model inconsistent; half a page works better than three pages.
• Keep the chatbot on your data. The switch next to Chatbot system role is on by default: the chatbot then answers from your instructions and your knowledge base. When it is off, the model also uses its general knowledge, which can lead to inaccurate or off-topic answers. Keep it on for a store.

Reset to Default restores the default role. A hand-over creates a ticket only when Enable Ticketing System is on in Account Settings → Tickets Settings, so write the escalation rule to match.

5Choose the AI model and search settings

Change the model when the analysis keeps reporting a model limitation, or when answers are too slow or cost too much. Set it in Chat Settings & Embed → AI model type. To compare models on your own data, open Test Chatbot → Compare Models: one question goes to a model from OpenAI, Anthropic, Google and xAI at the same time, with your chatbot's settings, and the four answers appear side by side. Each model can take 15–30 seconds. What to compare is explained in how to choose an LLM for customer support, and the AI benchmark shows how current models score on e-commerce questions.

The model list depends on your plan: plans with Chaterimo API keys offer a shorter list, plans with your own API key (BYOK: you use your own key and pay the model provider directly) offer every model.

Two more settings decide how the chatbot works with your data:

Search flow

Advanced RAG is the default and fits routine product questions, FAQs and customer support. RAG (retrieval-augmented generation) means the chatbot first searches your knowledge base and gives the passages it found to the model together with the question. Agentic search v2 lets the AI choose its own search steps and check the products it found; the app recommends it for complex products with many variants and parameters, where customers ask about compatibility. Its answers take longer and use more tokens. After you switch to it, Chaterimo prepares a catalogue index, usually within a minute; until then the chatbot uses the standard search.

Reasoning effort

On plans with your own API key, models that support it show Reasoning effort: how much the model reasons before it answers. none is the fastest and cheapest. Plans with Chaterimo API keys do not show this setting.

Select Save changes. Model, role and search flow apply to the next answer; they need no training.

6Retrain and test on Test Chatbot

After you change the knowledge base, open Knowledge Base → Knowledge Base and select Train chatbot (or Retrain). Entries waiting for training are marked Not trained. During the trial and on paid plans, new and changed data is also trained automatically every night, so training by hand only makes the change take effect sooner. Until training finishes, Test Chatbot warns that answers may be incomplete.

Then open Chatbot → Test Chatbot and ask the questions from your test set:

1. Select New conversation before each question, so earlier messages do not change the answer. If you have several chat widgets, choose the one to test under Chat widget.
2. Check the answer against the answer you expect: facts, products, tone and length.
3. Open Sources under the answer. It lists the products, pages, files and texts the answer was built from; select one to see its Source Details. If the right source is missing, fix the data (step 3). If it is there and the answer is still wrong, look at the role (step 4) or the model (step 5).
4. For an answer you cannot explain, select Analyze why AI answered like this (step 2).

Test Chatbot runs in testing mode: it creates no tickets, sends no ticket e-mails and its conversations do not appear in Chat History.

7Tune the proactive sales agent

The sales agent sends a proactive message: after a delay on the pages you choose, the chat window opens by itself with a message for the visitor. Rules are in Chatbot → Sales Assistant, on the Growth, Evolve, Advanced and Masterful plans; the number of active rules and AI messages per month depends on the plan (pricing).

Each rule has a Match Type (All pages, Only these pages or All except these pages), Page Patterns, a Delay (seconds) and a Message Type:

AI Generated

The AI writes the message from your AI Instructions and the page the visitor is on. It uses the AI model and language of the chat widget the rule belongs to and counts against your monthly AI messages.

Direct Message

Your own Message Text, shown at once and without AI cost. Direct messages are unlimited on all plans.

The app's own sales agent guide suggests starting with a delay of 30–60 seconds and using AI messages on specialised pages such as product pages, the cart and the checkout, not everywhere. Each rule fires once per visitor session. Change one thing at a time and compare the results of the rule, as described below.