Skip to main content
Concept

What is retrieval-augmented generation?

Letting a model look things up. Instead of answering from frozen memory, it fetches the relevant text first, then answers from that.

The short version

  • → RAG fetches relevant documents at question time and feeds them to the model as context.
  • → It lets a model answer from fresh or private data it was never trained on.
  • → Grounding answers in retrieved text reduces hallucination and lets you cite sources.

A base model answers from what it memorized during training, which is frozen and cannot include your private files or today's news. Retrieval-augmented generation, or RAG, fixes that by looking things up before answering.

The idea is simple: when a question comes in, first search a collection of documents for the relevant pieces, then hand those pieces to the model along with the question, and ask it to answer from them.

Retrieve, then generate

RAG has two steps. Retrieval searches your document collection for the passages most relevant to the question. Generation then feeds those passages to the model as context, so the answer is built from real, specific text rather than from hazy memory. The model still writes the answer; it just has the right material in front of it.

How the search finds the right text

Documents are usually turned into embeddings, numerical representations of meaning, and stored in a vector database. A question is embedded the same way, and the system pulls the passages whose meaning is closest. This finds relevant content even when the wording differs from the question.

Why it cuts hallucination

When the model is handed the actual source text and told to answer from it, it has far less need to invent. You can also show which passages the answer came from, so claims are checkable. It does not eliminate errors, but grounding answers in retrieved evidence makes them far more trustworthy.

RAG vs fine-tuning

Both add specialization, but they solve different problems. Fine-tuning changes how the model behaves. RAG changes what the model knows at answer time, and its knowledge updates the instant you update the documents. For "answer from this information", RAG is usually the right tool.

An analogy

A base model is a student answering a closed-book exam from memory. RAG turns it into an open-book exam: before answering, the student is handed exactly the right pages, so the reply is grounded in the source instead of half-remembered.

Go deeper

Embeddings

By the end you will know how a token becomes a vector, what "close" means between two vectors, how the numbers are learned, and what similarity can and cannot tell you. 17 min read →

Where Berges AI fits

RAG is the standard way to build an assistant over your own documents. Whether or not a given Berges AI feature uses retrieval, the principle is the same one it is built around: an answer grounded in real material beats a confident guess.

Related concepts

What is a context window?

The model's working memory: how much text it can hold in mind at once, and why it seems to forget once you go past it. Read →

What are AI agents?

The step from a model that answers questions to one that takes actions on your behalf, and why that step is harder than it sounds. Read →

What is fine-tuning?

Taking a capable general model and training it a little more on your own examples to make it better at one specific job. Read →

Things people ask

What problem does RAG solve?

It lets a model answer from information it was never trained on, such as your private documents or recent events, and it grounds answers in real sources so they can be cited and checked.

Is RAG better than fine-tuning?

For adding or updating knowledge, usually yes, because you just change the documents. Fine-tuning is better for changing behavior and style. Many systems use both.

Does RAG stop hallucination completely?

No, but it reduces it a lot. If retrieval surfaces the wrong passages, or the model strays from them, errors can still slip through. Showing sources lets you catch those.