What Is RAG? Retrieval-Augmented Generation Explained (and Its 6 Types)
Before the model answers, you fetch the facts and put them in the prompt. That one move fixes most of what makes an LLM useless on your own data — here's how.
TL;DR: RAG (Retrieval-Augmented Generation) is a simple idea with a lot of engineering behind it: before the model answers, you go and fetch the relevant facts, and you paste them into the prompt. That one move fixes most of what makes a general-purpose LLM useless on your own data — it stops inventing answers, it can cite where a claim came from, and you can update its knowledge by updating a database instead of retraining anything.
This guide covers: What RAG is · Why not fine-tuning · What happens inside the pipeline · The six types of RAG · Which one to start with · Where it breaks in production
Ask a general-purpose model about your own catalogue and you get one of two failures. Either it says it does not know, or — worse — it produces a fluent, confident answer about a product you have never sold. Neither is usable on a storefront.
The fix is not a bigger model. It is giving the model the right paragraph at the right moment. That is all RAG is.
What is RAG, in one plain sentence?
Retrieval-Augmented Generation is the practice of searching a knowledge source for relevant material and putting that material into the model’s prompt before it writes an answer.
The model is not taught anything new. Its weights never change. It is simply handed the facts as reading material, and asked to answer using them. Think of the difference between asking someone to recall a policy from memory and handing them the policy document open at the right page. Same person, very different reliability.
Why not just fine-tune the model instead?
This is the question every technical stakeholder asks in the first meeting, and it deserves a straight answer: fine-tuning and RAG solve different problems, and people reach for the wrong one constantly.
Fine-tuning teaches behaviour. RAG supplies facts. If you want the model to always reply in your brand’s tone, or to reliably output a particular JSON shape, fine-tuning is a reasonable tool. If you want it to know that SKU 44-A went out of stock this morning, fine-tuning is the wrong tool entirely — and an expensive way to be wrong.
| RAG | Fine-tuning | |
|---|---|---|
| Good for | Facts that change — stock, prices, policies, docs | Style, tone, output format, narrow task shape |
| Updating it | Write to a database. Live in seconds. | Rebuild a dataset and retrain. |
| Can it cite sources? | Yes — you know which chunks it read | No |
| Removing a fact | Delete the row | Hard. It is baked into the weights. |
In most commercial projects the honest answer is RAG first. It is cheaper, it is inspectable when it goes wrong, and it does not strand you with a model that needs retraining every time a policy changes. Fine-tuning, if you need it at all, comes later and sits on top.
What actually happens inside a RAG pipeline?
It helps to see RAG as two separate pipelines that run at different times. Most of the engineering effort — and most of the failures — live in the first one, which is the one nobody demos.
Two details in that diagram do more damage than anything else when they are done carelessly:
- Chunking. Split a shipping policy in the middle and neither half answers the question. Split a product spec table by rows and you get half a specification. Chunk boundaries decide what it is possible for the system to retrieve.
- Embedding. Vectors capture meaning, not exact strings. That is a strength for “can I return this” matching a refunds policy that never uses the word “return”. It is a weakness for SKUs, part numbers and model codes, where you actually wanted an exact match — which is why serious systems combine both.
What are the types of RAG?
“RAG” describes a family, not a single architecture. The variants below are roughly a maturity ladder: each one exists because the previous one broke somewhere specific. You do not need the fanciest one. You need the cheapest one that survives your actual questions.
| Type | The problem it solves | Cost of adding it |
|---|---|---|
| Naive RAG | Getting grounded answers at all, from a standing start | Low — days |
| Advanced RAG | Retrieval that returns plausible-but-wrong passages | Moderate — adds latency |
| Modular RAG | A pipeline nobody can safely change six months in | Design effort, not runtime |
| Graph RAG | Questions that need connected facts, not one passage | High — you must build the graph |
| Agentic RAG | Questions one search can never answer | High — slower, costlier per query |
| Multimodal RAG | Answers that live in images, scans and tables | Depends on your data |
Which type should you actually start with?
Start with the simplest one and let real questions push you up the ladder. That advice sounds conservative; it is really just cheaper. Every rung you climb adds latency, cost per query, and something new that can fail silently at 2am.
A workable sequence:
- Build naive RAG on a narrow, high-value corpus. One document set, one audience. Shipping and returns policies are a good first target — small, stable, and responsible for a large share of support tickets.
- Collect the questions it gets wrong. Not vibes — an actual list. This is the single most valuable artefact of the whole project, and almost everyone skips it.
- Read the failures and diagnose the stage. Did the retriever return the wrong passage, or did it return the right passage and the model ignored it? Those are different bugs with different fixes, and the distinction tells you which type of RAG you actually need next.
- Add exactly the capability those failures demand. Reranking for near-miss retrieval. A graph if answers span multiple entities. Agentic loops only when a single query genuinely cannot get there.
We would rather ship a boring pipeline that answers 200 real questions correctly than an elegant one that impresses in a demo and quietly fails on the long tail.
Where does RAG go wrong in production?
Prototypes are easy. The gap between a working demo and something you would put in front of customers is mostly made of these:
- Retrieval returns something plausible but wrong. The model then writes a confident answer on a bad source. This fails worse than saying nothing, because it looks correct.
- Stale index. The document changed; the index did not. Now the system is authoritative and out of date — a particularly nasty failure for prices and stock.
- Permission leakage. If your index does not carry access rules, retrieval will happily surface a document the asker was never allowed to see. Filter at query time, not after generation.
- No evaluation. If you cannot say whether last week’s change made retrieval better or worse, you are not engineering — you are decorating. A fixed set of question/answer pairs you re-run on every change is the minimum bar.
- Silent context truncation. Stuff too much in and the important passage falls off the end of the window. The answer degrades and nothing logs an error.
None of these are exotic. They are the ordinary consequences of treating RAG as a prompt problem rather than a retrieval-and-data problem, which is what it actually is.
Frequently asked questions
Is RAG the same as giving ChatGPT a document to read?
It is the same idea, automated and scaled. Pasting a document into a chat window is manual retrieval — you decided what was relevant. RAG does that selection automatically, per question, across a corpus far too large to paste, and does it in milliseconds.
Does RAG stop the model from hallucinating completely?
No, and be sceptical of anyone who says otherwise. RAG dramatically reduces invented facts because the model has the real passage in front of it, and it makes errors traceable to a source you can inspect. But a model can still misread a retrieved passage or fill a gap when retrieval comes back empty. Grounding plus evaluation gets you far; grounding alone is not a guarantee.
Do I need a vector database to do RAG?
Not necessarily. For a few thousand chunks, a vector column in Postgres or even an in-memory index is genuinely fine and much simpler to operate. Dedicated vector databases start paying for themselves at larger scale, with heavy filtering, or when you need frequent live updates. Choosing one on day one is a common way to add operational burden before you have a problem worth solving.
How much data do I need before RAG is worth it?
Less than people expect. The threshold is not volume, it is whether the answers exist in writing somewhere and change often enough that maintaining them by hand is painful. A 40-page policy set that changes quarterly is a perfectly good RAG use case. Ten static FAQs are not — just put them on a page.
Can RAG work over a live product catalogue?
Yes, and it is one of the strongest commercial uses, but treat volatile fields carefully. Descriptions and specifications sit fine in the index. Price and stock should be fetched live at answer time rather than embedded, otherwise you will confidently quote yesterday’s price. In practice that means a hybrid: retrieval for the wording, a direct query for the numbers.
Can Ecarter build this for us?
Yes — RAG systems, AI assistants and catalogue-aware search are part of our AI development and LLM development work, and we integrate them into CS-Cart, Magento and Shopify stores as well as standalone applications. We usually start with a narrow, measurable use case rather than a platform.
Thinking about putting your own documents or catalogue behind an AI assistant? Talk to Ecarter about a RAG pilot scoped to one real use case — and read on for the six types in detail.
Nisha Gaur is a Technical Content Writer at Ecarter Technologies. She writes technical documentation, tutorials and buying guides covering CS-Cart, Magento, Shopify and eCommerce development.