By Damian Siembor · Published 31 October 2026

Fine-tuning vs RAG: a 10-minute check before you spend anything

Fine-tuning changes how a model answers; RAG changes what it can look up. A short checklist to decide, with real runs where fine-tuning helped a lot, a little, and where it could not add knowledge.

"Should we fine-tune or use RAG?" is usually asked as a technology question. It is really a question about what is wrong with the model's answers today. If the model has the wrong information, you need to give it information. If it has the right information in the wrong shape, you need to change how it answers. Those are different tools, and picking the wrong one costs weeks.

We sell fine-tuning, so you should know our bias, and we will say it plainly: a lot of projects that arrive asking for fine-tuning need retrieval instead, and this page is written to help you find that out before you pay for anything.

What each one actually changes

RAG (retrieval-augmented generation) leaves the model alone. At question time, you search your documents, paste the best passages into the prompt, and ask the model to answer from them. The model's knowledge stays the same. What changes is what is in front of it. Updating the knowledge means updating the index.

Fine-tuning changes the model's weights using examples of inputs and the outputs you want. It reliably teaches format, tone, length and task habits. It is a weak and unpredictable way to store facts: a model trained on your documents may pick up some of what they say, may mix it with what it already believed, and gives you no source to check. When the documents change, you train again.

The 10-minute check

Answer these five questions about your use case, in order. Stop at the first one that gives you an answer.

1. Does a good prompt already work? Write the prompt with the format spelled out and two or three worked examples, and run 20 real inputs. If 18 are fine, you do not have a problem to spend money on. This is also your baseline for everything below.
2. Does the right answer depend on a document that changes? Prices, policies, tickets, product specs, anything with a date on it. If yes, you need retrieval. Fine-tuning will not know last month's price list.
3. Do the wrong answers have the right facts in the wrong shape? The model answers correctly but wraps JSON in prose, writes five paragraphs where you want one line, uses the wrong label set, or ignores your house style. That is a fine-tuning problem.
4. Can you produce 500 examples of the exact answer? If you cannot write them, or if two people on your team would write them differently, the model will inherit the disagreement. Fix the task definition first.
5. Do you need both? Often yes: retrieval supplies the facts and a tuned model answers in the shape you need. Do retrieval first, because it is cheaper to change, then fine-tune only if the format or tone is still wrong. A tuned model can then be given the passages in its prompt.

If you go the retrieval route, a plain grounded prompt is a good start and nothing here needs a tuned model:

Answer using only the context below. If the answer is not in the context, say you do not know.

Context:
<passage 1>
<passage 2>

Question: <the user's question>

Two real runs that show the limit

Two of the public finetuna reports are cases where it is easy to expect more than fine-tuning delivers.

The Polish medical exam run trained Bielik 11B on 1,297 exam questions with written explanations, 2 epochs (full passes through the file), 129.3 GPU minutes. Validation loss (how far the model's answers are from yours on held-out examples, where lower is better) went from 1.038 to 0.439. On the 19 held-out questions with a known answer, the base model picked the right letter 8 times and the tuned model 10 times. That is two more correct answers out of nineteen, not evidence that the model learned medicine. What changed clearly is the style: the tuned model reasons in the dataset's pattern and its answers run long, and 4 of the 19 hit the length limit before reaching the letter. It is a good example of what the loss number does and does not tell you.

The Italian instructions run used 9,421 examples on Mistral Nemo 12B, 2 epochs, 136.2 GPU minutes: validation loss 1.539 to 1.121, and base and tuned answers read alike, because Nemo already writes fluent Italian. The tuned model picked up the dataset's habits, including a flaw: it repeats captions inside one list of ten, which the base model does not. A fine-tune imports the quirks of its data along with the good parts.

Compare the JSON extraction run, the kind of task fine-tuning is for: a fixed output scheme. With the same prompt the base model produced valid JSON but filed entities under the wrong categories. The tuned one learned the dataset's scheme (8 of 20 exact matches against 1, and entity F1, a score from 0 to 1 for finding the right entities, 0.71 against 0.25). The training did not change what the model knew. It changed what it did with it.

Mistakes we see

• Training on a manual to make a chatbot "know" it. You get a model that sounds like the manual. Put the manual behind retrieval.
• Skipping the baseline. Without 20 real inputs run through a good prompt first, you cannot tell whether training helped.
• Judging by loss. A lower validation loss only means the model predicts your held-out examples better. Read the answers.
• Mixed output formats in the data. The model learns all of them and uses them at random.

If the check says fine-tune

The cheapest way to find out is a small run: package S is $5 and covers up to 20M training tokens (a token is a piece of a word, about 3/4 of a word in English), and the three public runs above all fit inside it. Start from thesmall-model fine-tuning guide for dataset format and epochs, and read the result with the report's held-out answers before you commit more. If you want to run what you get, see the GGUF and Ollama guide.

finetuna does not do retrieval and does not host models: it trains and gives you files. If your check lands on RAG, there is nothing here to buy, and that is the right answer.