By Damian Siembor · Published 31 October 2026

How to fine-tune a small language model: data, epochs, results

A practical walk-through: scope a narrow task, write the JSONL, pick epochs, read the validation loss and the held-out answers, and know when not to fine-tune. With three real runs.

"Small" here means the open models you can fine-tune on one 24 GB GPU: 8B to 14B parameters, and 32B with a shorter context. The catalog on finetuna runs from Qwen3 8B to Qwen3 32B. If you need something under 8B, this service is not the tool. The method in this guide is the same, only the hardware and the runner differ.

Fine-tuning is good at one thing: making a model answer in a fixed shape, tone or style, every time. It is bad at teaching a model facts that change. Keep that line in mind, because most of the decisions below follow from it. Thefine-tuning vs RAG check covers the facts side.

1. Scope one narrow task and get a baseline

Write the task as one input and one output: a sentence in and a JSON object out, a review in and one word out, or a schema and a question in and a query out. A narrow task gives a dataset you can check by eye and a result you can measure.

Before you train anything, run 10 to 20 real inputs through the base model with the best prompt you can write, including two or three worked examples. If that already works, stop: a prompt costs nothing and fine-tuning does not fix a task you cannot yet describe. If it fails in a consistent way (wrong format, wrong labels, chatty preamble), you have a good fine-tuning case, and you have also written your first test set.

2. Write the dataset: one example per line

The input is a JSONL file, one JSON object per line. finetuna accepts three shapes, and every line is checked in your browser before anything is uploaded. The first, chat messages, is the most general (each example is one line in the file, so no line breaks inside the JSON):

{"messages": [{"role": "system", "content": "Reply with the rewritten sentence only."}, {"role": "user", "content": "You must trust your instincts."}, {"role": "assistant", "content": "Trust your instincts, you must."}]}

The two shorter forms are converted into the same thing:

{"prompt": "Review:\nThe battery died after a week.", "completion": "negative"}
{"instruction": "Extract the product name.", "input": "I returned my AeroPress yesterday.", "output": "AeroPress"}

Rules that save a failed job:

• The last message must come from the assistant. A messages list needs at least a user and an assistant turn.
• Be boringly consistent in the output. If half the answers wrap JSON in a code fence and half do not, the model learns both.
• If you want a system prompt to be part of training, put it in the rows. The "system prompt" field on the new-job page is written into the Modelfile for Ollama and is not used during training.
• Remove duplicates and near-duplicates yourself. The validation split is a random 5% of your file, held back from training so the model can be tested on it. A near-copy of a validation example in the training part makes the loss (how far the model's answers are from yours, where lower is better) look better than it is.
• Examples longer than the sequence length are cut, so keep each example under it. The limit is 2048 tokens, or 1024 for Qwen3 32B. A token is a piece of a word, about 3/4 of a word in English.

The line above is an illustration written for this page, not a row from a published dataset.

3. How many examples, how many epochs

An epoch is one full pass through your file. There is no honest universal number of examples or epochs. The six public reports on finetuna range from 720 to 9,421 examples. What we can say from running three of them is shown below. All three runs in this table fit the $5 package S, which covers up to 20M training tokens (the tokens in your file times the number of epochs).

The new-job page gives one rule of thumb, and it is only a rule of thumb: 1 epoch for large datasets, 2 to 3 for datasets under about 2,000 examples. More epochs on a small file help it learn the pattern, and past a point they just memorise it, which is why the held-out answers in step 5 matter more than the loss curve.

4. What the training does, and what you cannot change

Every finetuna job is QLoRA. The base model is held in a 4-bit form (a quantization: a smaller, slightly less accurate copy of the model). Small LoRA adapters of rank 16 and alpha 32 (add-on weights, the only part that is trained) are trained on top of it, at a learning rate of 0.0002 with a cosine schedule, 3% warmup and batch size 8. 5% of your examples, at most 500, are held out for validation. You do not tune these numbers on the service. If you need to, run the same recipe yourself with Unsloth or PEFT. TheLoRA vs QLoRA guide explains what each setting does.

5. Read the result: validation loss and the held-out answers

The report gives the validation loss before and after training, a loss curve, and up to 20 held-out prompts answered by both models side by side. Use them in that order of trust: the answers first, the loss second.

A large loss drop is expected when the dataset teaches a format the base model does not know. The Yoda run dropped from 4.389 to 0.552 because the base model wraps its answers in quotes and picks its own word order. A small drop is a warning, not a failure: the Italian instructions run went from 1.539 to 1.121 on 9,421 examples, and its answers read much like the base model's, because the base model already wrote fluent Italian.

Put numbers on the answers when the task allows it. On the JSON extraction run the base model produced valid JSON on all 20 held-out sentences. It matched the reference exactly on only 1. After tuning it matched 8 of 20, and entity F1 went from 0.25 to 0.71. The remaining misses are mostly entity boundaries, such as a trailing full stop. Twenty prompts is a smoke test, not a benchmark: use it to decide whether to look further, then test on your own data.

When not to fine-tune

• The answer depends on documents or prices that change. Use retrieval. A tuned model will still not know last month's price list.
• A good prompt with a few examples already works.
• You cannot write down what a correct answer looks like. A fuzzy dataset gives a fuzzy model.
• Your data has personal data or is under NDA and you cannot use a Secure GPU. The default Community GPU is for data you can share.
• You need a full fine-tune, preference tuning, a model under 8B or hosted inference. finetuna does QLoRA on 8B to 32B models and gives you files. It does not host the model.

If a job fails on our side, we rerun it. If it fails a second time, we refund the full price. If it succeeds but the result disappoints, there is no refund.