Teams usually ask which one to pick. The better question is what each one is for. Retrieval gives a model facts it did not have at training time, and you can change those facts this afternoon. Fine-tuning changes how the model behaves: the format, the vocabulary, the judgement calls it makes on your kind of problem. A wrong answer caused by a missing document is a retrieval problem. A right answer in the wrong shape is a fine-tuning problem.
| What matters | RAG | Fine-tuning |
|---|---|---|
| What it changes | What the model knows at the moment you ask | How the model answers, in tone, format and judgement |
| Updating it | Re-index the document. Live in minutes. | Re-train and re-evaluate. Days, and a new version to ship. |
| Where your data sits | In your index, retrievable and deletable per document | Absorbed into the weights and not separable afterwards |
| Showing the source | Every answer can cite the passage it came from | No citation; the answer comes out of the weights |
| Cost profile | Low to set up, paid per query on retrieval and context | Higher up front, cheaper per query afterwards |
| What it fails at | Cannot fix an answer that is well-sourced but badly shaped | Cannot know a document written after training |
| GDPR erasure | Delete the document and the answer stops appearing | Erasure means re-training, because the data is in the weights |
When retrieval is the answer
- The facts change: prices, policies, stock, contracts, anything with a version.
- You need every answer to cite where it came from.
- A person must be able to demand deletion and have it actually take effect.
- You are starting, and you want something useful in weeks rather than months.
- The knowledge is large and the questions only touch a little of it at a time.
When fine-tuning earns its cost
- The answers are right and the shape is wrong: your format, your terminology, your house style.
- The task is narrow and repeated thousands of times a day, where per-query cost dominates.
- You have enough labelled examples of the behaviour you want, and a way to tell better from worse.
- A smaller model, tuned, would do the job a large one is doing expensively.
- The judgement you need is specific to your domain and no prompt reliably reproduces it.
Start with retrieval. It is faster to stand up, easier to correct, and it tells you what the system is actually getting wrong. Most of what teams plan to fix with fine-tuning turns out to be a retrieval problem or a prompt problem. When you do fine-tune, fine-tune on top of a working retrieval system rather than instead of one: the model learns the shape, and retrieval keeps the facts current.
Two weeks of AI Discovery ends with the answer for your case, the systems it touches, and what it would take. Or take the readiness check first. Take the readiness check
What buyers ask before they choose.
Start with retrieval. It is faster to stand up, easier to correct, and it shows you what the system is actually getting wrong. Much of what teams plan to fix with fine-tuning turns out to be a retrieval problem or a prompt problem.
Not reliably, and not in a way you can update. Fine-tuning shapes how a model answers: the format, the vocabulary, the judgement. Facts belong in retrieval, where you can change them this afternoon and see the answer change with them.
Not without training it again. Once data is in the weights it is not separable, so an erasure request means rebuilding the model. In a retrieval system you delete the document and the answer stops appearing. If the right to erasure applies to your data, that difference decides the architecture.
With retrieval, every answer can cite the passage it came from, which is how a reader checks it and how your team debugs it. A fine-tuned model produces the answer out of its weights with no source to point at.
When the answers are right and the shape is wrong, when the task is narrow and repeated thousands of times a day so cost per query dominates, and when you have enough labelled examples plus a way to tell better from worse. Fine-tune on top of a working retrieval system, not instead of one.
Bring us the problem nobody has cracked yet.
We are a small team of senior specialists. We pick the right model and the right layer, and we build the least machinery that does the job. You get a call with an engineer, not a sales deck.