By Damian Siembor · Published 31 October 2026

LoRA vs QLoRA fine-tuning: what changes in practice

What LoRA and QLoRA actually do, when each makes sense, the exact recipe finetuna runs (rank 16, alpha 32, learning rate 2e-4), and what happens between the adapter and the GGUF.

Both methods train a small add-on instead of the whole model. They differ in how the base model is held in memory while that happens. The difference matters for which GPU you need, much less for what you get back. This guide explains both, then shows the exact recipe finetuna runs, including what happens between the adapter and the GGUF file.

What LoRA does

LoRA (low-rank adaptation, Hu et al., 2021, arXiv 2106.09685) freezes the base model's weights. For selected weight matrices it learns a pair of thin matrices whose product is the update. The width of that pair is the rank (r). Because only the thin matrices are trained, the optimizer state is small and the result is a compact file, the adapter, which is added to the base weights at run time or merged into them once.

Two numbers define it. The rank r sets how much the adapter can change: higher is more capacity and more parameters. The scale alpha sets how strongly the learned update is applied. In the standard formulation it is multiplied by alpha / r. The third knob that matters most is the learning rate. LoRA runs typically use a much larger one than full fine-tuning does.

What QLoRA adds

QLoRA (Dettmers et al., 2023, arXiv 2305.14314) keeps the frozen base model in 4 bits during training. That is a quantization: a smaller, slightly less accurate copy of the model. The paper uses a 4-bit NormalFloat format with double quantization, and does the arithmetic in bfloat16. The adapters are still trained at higher precision. The authors reported that this matched 16-bit fine-tuning on their benchmarks and made a 65B model trainable on a single 48 GB GPU. The cost is some speed, because 4-bit weights are expanded as they are used.

Which one should you use

If the model fits in 16-bit on the card you have, plain LoRA is the simpler choice. QLoRA is what lets a 24 GB card train models in the 8B to 32B range. That is why every finetuna job is QLoRA on an RTX 4090: our model list notes say Qwen3 14B fits at sequence length 2048 and Qwen3 32B fits in 4-bit at sequence length 1024. Sequence length is the longest example the model sees, counted in tokens. A token is a piece of a word, about 3/4 of a word in English. We have not run a head-to-head of LoRA against QLoRA, so this page makes no quality claim between them beyond what the papers report.

The exact recipe finetuna runs

Every report ends with a configuration table. This is the one from theJSON extraction report, row for row:

Across the six reports only two values change: the epochs (an epoch is one full pass through your file: 1 for the sentiment run, 3 for Yoda, 2 for the other four) and the base model licence (Llama 3.1 Community License for the Yoda run, Apache 2.0 for the rest). The rest is identical. A few settings are not in that table. They come from our model catalog and our training script, not from the reports:

On the service you choose the base model, the epochs (1 to 3) and the GPU tier. The rest of the recipe is fixed. That is a deliberate limit: one tested recipe, a predictable price, no sweeps. If your task needs a different rank or learning rate, a DIY run with Unsloth or PEFT is the right tool, and the adapter you get from finetuna works with PEFT and Unsloth.

From adapter to GGUF: where the quantization happens twice

The 4-bit weights used during training are not what you get back. After training, the adapter is merged into the base model in 16-bit, the merged model is converted to a 16-bit GGUF, and that file is quantized to the two sizes in the delivery:

python convert_hf_to_gguf.py merged/ --outfile model-f16.gguf --outtype f16
llama-quantize model-f16.gguf model-q4_k_m.gguf Q4_K_M
llama-quantize model-f16.gguf model-q8_0.gguf Q8_0

So the q4_k_m file is a fresh 4-bit quantization of the merged model, in llama.cpp's format, not the training-time one. It is the smallest file and the one the Modelfile points at: about 5 GB for an 8B model. q8_0 is about 9 GB and closer to full precision. If quality is the point and disk is not, edit the FROM line to use q8_0. TheGGUF and Ollama guide goes through running it.

What this looked like on real runs

The text-to-SQL run is one of the heavier published runs: 3,000 examples, 2 epochs, 3.74M training tokens, 714 steps, 85.8 GPU minutes on Qwen3 8B. Validation loss (the loss on examples held out from training: how far the model's answers are from yours, where lower is better) went from 1.415 to 0.114. On 20 held-out questions the tuned model matched BIRD's reference query character for character 5 times and the base model 0 times. That number undersells both: several base answers are correct SQL written in a different style, and exact match is not execution accuracy, which we could not measure without the BIRD databases.

The JSON extraction run is the cheaper end: 3,000 shorter examples, 2 epochs, 714 steps, 24.6 GPU minutes, same model, same recipe. The recipe did not change between them. The data length did.

Limits worth knowing

• An adapter changes behaviour and style well. It is a poor way to add large amounts of new knowledge, as the fine-tuning vs RAG guide explains.
• This is supervised fine-tuning only. There is no preference tuning, no full fine-tune and no continued pretraining.
• A low loss is not a good model. It measures how well the model predicts your held-out examples, and a base model that does not know your format will always start high.
• Gated base models (Llama 3.1, Mistral Nemo, Gemma 3) need their licence terms respected in what you ship. Llama derivatives carry "Built with Llama".