If someone has proposed “a RAG system” and you nodded along, this is for you.
RAG in one sentence
Retrieval-augmented generation means that before an AI answers, it looks up relevant passages from your documents and uses them to write its answer.
Think of a new employee taking a closed-book exam versus an open-book one. A plain LLM answers from memory. RAG hands it the right pages first.
Why it matters
LLMs do not know your policies, products or contracts. Retraining a model on your data is costly and rarely necessary. RAG lets you keep your data separate, update it daily, and cite sources for each answer.
How it works
- Ingest. Documents are cleaned and split into passages.
- Index. Passages are stored for both keyword and meaning-based search.
- Retrieve. A question finds the best passages.
- Generate. The model writes an answer using only those passages, with citations.
Where RAG goes wrong
Bad retrieval. Most quality problems are search problems. If the right passage is not found, the answer cannot be right.
Messy sources. Tables in PDFs, scanned pages and duplicated old versions confuse systems. Old documents can give confident answers that used to be right.
Permission leaks. If the assistant can see salary files, anyone who asks might too. Permissions must be applied at search time.
No evaluation. Without a set of test questions, nobody can tell whether a change helped.
Five questions to ask your team
- Where do the answers come from, and can users see the source?
- What happens when the documents do not contain the answer?
- How are document permissions enforced?
- How do outdated documents get removed?
- How do we measure accuracy, and what was it last week?
If those get crisp answers, you are in good shape.
When not to use RAG
If the question needs totals or live numbers, such as “what was revenue last quarter?”, you want a query against your database, not document search. Good assistants route between both.
For the technical design, download the RAG Architecture Blueprint or see our RAG development service.