What Is RAG (Retrieval-Augmented Generation)?
How AI systems answer from your own documents, and why it is the default for business use.
What is retrieval-augmented generation?
Retrieval-augmented generation, or RAG, is a technique where a system searches your own documents for relevant passages and gives them to a language model along with the question. The model answers from that supplied material rather than from memory alone, which makes answers specific to your organisation and allows sources to be shown.
Almost every useful business AI application that answers questions about your own information uses this approach. Understanding it makes the difference between specifying something achievable and something that will disappoint.
Why is RAG used instead of training a model?
Training a model on your data requires substantial expertise, data volume and budget, and the result is fixed at training time. Retrieval works with existing models, updates the moment a document changes, and lets the system cite the passage it used.
For almost all business cases, retrieval is the right default. Training becomes relevant only at scale with a specific requirement retrieval cannot meet.
Does RAG stop AI making things up?
It reduces it substantially by grounding answers in real passages, and showing sources lets users verify. It does not eliminate the risk — a model can still misread a passage or combine two incorrectly, particularly when the retrieval returns weak matches.
Instructing the system to answer only from retrieved content and to say when it cannot find an answer meaningfully improves behaviour, and should be tested rather than assumed.
What does it need to work well?
Current, accurate source documents, sensible chunking so passages retrieved are coherent, a retrieval method that actually finds the right passages, and permission handling so users only receive content they may see. Source quality matters more than model choice.
A retrieval system over outdated documentation will answer confidently and wrongly. Keeping the source current is the main ongoing maintenance task.
How is permission handled?
Retrieval must filter to documents the asking user is allowed to read, before passages reach the model. If this is added after the fact rather than designed in, a system can quote confidential material to someone who should not see it.
Ask any supplier specifically how permissions are enforced at retrieval time. It is the most common serious design flaw in internal AI tools.
What does it cost to run?
Embedding and storage costs for the document index, plus per-request model costs that scale with usage and with how much retrieved content is sent. Larger retrieved context means better answers and higher cost per question.
Model this at expected question volume. Costs are modest at pilot scale and become meaningful once an organisation uses the system daily.
Services related to this topic
Related solutions
More on ai
Related comparisons
Want help applying this?
Tell us your situation and we will tell you what we would actually do first.