RAG (Retrieval-Augmented Generation)

RAG (retrieval-augmented generation) means the system searches your documents for relevant passages first, then hands those passages to the model along with the user's question so the answer is grounded in your content instead of the model's memory. Search, then answer. That is it.

Why it matters when you are buying or building

Nearly every "train the AI on your website" feature you will see demoed is RAG. The model is not being trained on anything. Your pages are being chunked, indexed, and looked up at question time. That distinction matters commercially, because it means updates are cheap and instant. Change a page, reindex, done. No retraining, no waiting.

It also tells you where to spend effort. RAG quality is mostly a content problem, not an AI problem. If your pricing lives in a PDF table that chunks badly, or three pages contradict each other on service hours, retrieval will surface the wrong passage and the model will faithfully repeat it. Garbage retrieved, garbage generated.

Things worth checking before you sign:

• Can you see which chunks were retrieved for a given answer? Without that, debugging a wrong answer is guesswork.
• Can you scope retrieval per site, per brand, or per tenant?
• Does the agent get told what to do when retrieval comes back empty?

That last one is the difference between "I do not have that, let me take a message" and a confident invention. See hallucination.

What I have seen in production

For a jewelry retailer I run a shopping assistant that answers across multiple sites, chat and voice, off a shared knowledge base. The wins there came from content hygiene, not model tuning: splitting one enormous FAQ page into focused pages, deleting outdated policy copy that kept getting retrieved, and making sure product care instructions were written as prose rather than as image captions the indexer could not read.

I also build demo agents that scrape a prospect's website into a knowledge base in a couple of minutes, which is a genuinely good sales motion. It is also a good honesty test. If the prospect's site is thin or contradictory, the demo agent shows it immediately. That is useful information for both of us.

One practical limit: RAG is for reference content, not live state. Order status, appointment slots, and account balances should come from an API call, not from indexed pages. Retrieval will happily return a six month old snapshot and nothing in the pipeline knows it is stale. Use retrieval for what is true in general and function calling for what is true right now. Getting that split right is most of the work in a knowledge base build.