Stop Prompting, Start Tuning: A Developer's Guide to LoRA, QLoRA and the Dataset Mistakes That Ruin Both

Quick Answer

Fine-tuning changes a model's weights so it follows your format, vocabulary and task more reliably, while prompting only changes what the model is shown. For most teams the practical route is parameter-efficient tuning: LoRA trains small adapter matrices on top of frozen weights, and QLoRA does the same on a 4-bit quantized base model so it fits on far less GPU memory. The step that decides success is not the training run. It is the evaluation you write before it and the dataset you build for it. This guide walks through a seven-step pipeline with working code, then covers the dataset mistakes that ruin both LoRA and QLoRA runs.

When Is Fine-Tuning Worth It?

Provider guidance is consistent on the order of work: write evaluations first, improve prompts and retrieval next, and tune only if a measurable gap remains. OpenAI's own optimization guide says prompt engineering "may be all you need" for many use cases. Tuning tends to pay off for narrow, repeated tasks such as classification, fixed output formats, tone and domain terminology. It suits fast-changing facts less well, and retrieval is usually the better fit there.

One platform note before the code. OpenAI announced in May 2026 that it is winding down its self-serve fine-tuning platform, so a new reader generally cannot start there. This guide therefore uses open-weight models with Hugging Face tooling. The same steps apply when you tune through a managed service such as Amazon Bedrock or Google's Vertex AI, where the dataset and evaluation work is identical and only the training call changes.

For a non-technical view of why companies are tuning models at all, including what Coinbase reported this month, see this news-style analysis of the evidence on fine-tuning versus prompting.

The Seven-Step Fine-Tuning Pipeline

1. Write the evaluation first

Collect real examples of the task and split them before you do anything else. Carve out a holdout set from the same collection as your training data, because OpenAI's guide describes this holdout as a control group for evals. Keep its mix of inputs and outputs roughly as diverse as the training set. Then score your current prompt-based or retrieval-based system on it. That score is the baseline every tuned model must beat, and without it you cannot show that tuning helped.

2. Build the dataset

For chat-style models, the usual format is JSONL with one complete JSON object per line. TRL's SFTTrainer accepts conversational data in a messages layout and applies the model's chat template automatically.

{"messages": [{"role": "system", "content": "You label support tickets."}, {"role": "user", "content": "I was charged twice for my order."}, {"role": "assistant", "content": "billing"}]}
{"messages": [{"role": "system", "content": "You label support tickets."}, {"role": "user", "content": "The app crashes when I upload a photo."}, {"role": "assistant", "content": "bug"}]}

How many examples? OpenAI's guide sets a floor of 10, reports improvements from 50 to 100 well-crafted examples and suggests starting with 50. Those numbers are for a hosted service and a simple task, so treat them as a starting point and let your own holdout score decide when you have enough. If 50 careful examples change nothing, the same guide advises rethinking the task or prompt before adding more data.

3. Pick LoRA or QLoRA

LoRA freezes the pre-trained weights and trains small low-rank matrices inserted into the transformer layers. The original paper reports up to 10,000 times fewer trainable parameters and about 3 times less GPU memory than fully fine-tuning GPT-3 175B with Adam. QLoRA keeps the base model frozen in 4-bit precision and backpropagates through it into the adapters. Its authors report tuning a 65 billion parameter model on a single 48 GB GPU using the NF4 data type, double quantization and paged optimizers.

A fair caveat comes from the paper "LoRA Learns Less and Forgets Less": in standard low-rank settings LoRA underperforms full fine-tuning on the target domain, but it better preserves the base model's behavior on other tasks. Choose it for cost and stability, not because it is identical to full tuning.

The configuration below uses example values. In the PEFT library the defaults are r=8, lora_alpha=8 and lora_dropout=0.0, and target_modules="all-linear" applies the adapter to every linear layer except the output layer.

import torch
from transformers import AutoModelForCausalLM, BitsAndBytesConfig
from peft import LoraConfig

model_id = "your-org/your-open-weight-base-model"  # example placeholder

bnb = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
    bnb_4bit_compute_dtype=torch.bfloat16,
)
model = AutoModelForCausalLM.from_pretrained(model_id, quantization_config=bnb)

lora = LoraConfig(
    r=16,                     # example value; PEFT default is 8
    lora_alpha=32,            # example value
    lora_dropout=0.05,        # example value; PEFT default is 0.0
    target_modules="all-linear",
    task_type="CAUSAL_LM",
)

Drop the quantization_config argument to run plain LoRA on a 16-bit model if you have the memory.

4. Train with SFTTrainer

TRL's SFTTrainer handles both standard and conversational datasets. Two settings deserve attention. First, assistant_only_loss=True computes loss only on assistant replies, but it needs a chat template that marks assistant spans with {% generation %} keywords, and TRL patches some model families for this. Second, the end-of-sequence token must match the chat template, for example <|im_end|> for Qwen-style templates.

from datasets import load_dataset
from trl import SFTConfig, SFTTrainer

train = load_dataset("json", data_files="train.jsonl", split="train")
evalset = load_dataset("json", data_files="holdout.jsonl", split="train")

args = SFTConfig(
    output_dir="ticket-labeler-lora",
    num_train_epochs=3,                # example value
    per_device_train_batch_size=4,     # example value
    learning_rate=2e-4,                # example value
    eval_strategy="steps",
    eval_steps=50,
    assistant_only_loss=True,          # needs {% generation %} markers in the template
)

trainer = SFTTrainer(
    model=model,
    args=args,
    train_dataset=train,
    eval_dataset=evalset,
    peft_config=lora,
)
trainer.train()
trainer.save_model()

Call print_trainable_parameters() on the PEFT model to see how little is being trained. PEFT's own quick-start example reports about 0.04 percent of a 1 billion parameter model.

5. Evaluate against the baseline, and for forgetting

Run the tuned model on the same holdout set you used for the baseline and compare scores. Then check what you may have broken. A task-specific holdout measures the gain, while a general benchmark suite shows regressions on everything else. The lm-evaluation-harness can load a base model plus a LoRA adapter through the peft= argument.

lm_eval --model hf \
  --model_args pretrained=your-org/your-open-weight-base-model,peft=./ticket-labeler-lora \
  --tasks hellaswag,arc_easy,boolq \
  --device cuda:0

Run the same command without peft= to get the base model's numbers, and compare the two. If the holdout score rises but general scores fall sharply, reduce epochs, lower the learning rate or lower the rank.

6. Secure it

Fine-tuning adds new ways to fail. Four risks are worth writing into your checklist, using the OWASP Top 10 for LLM Applications (2025) as the reference:

• Sensitive information disclosure (LLM02): personal data and secrets that enter the training set can be reproduced by the model. Carlini and colleagues showed extraction of names, phone numbers and email addresses from GPT-2.
• Supply chain (LLM03): OWASP warns that a malicious LoRA adapter can compromise a base model, so download adapters and base weights only from sources you trust.
• Poisoning (LLM04): data used in fine-tuning can be manipulated to add backdoors or bias, so know where every example came from.
• Safer file formats: Hugging Face documents that loading a pickle file can execute arbitrary code, and safetensors was built as a safe alternative. Prefer safetensors for weights you did not create yourself.

Safety behavior can also erode. Qi and colleagues showed that adversarially designed fine-tuning with only 10 examples jailbroke GPT-3.5 Turbo, and that even benign datasets can weaken alignment to a lesser extent. Re-run your safety tests after every tuning job, not only the accuracy tests.

7. Serve and monitor

For serving, PEFT can merge the adapter into the base weights so inference carries no extra adapter latency. One detail catches people out: merge_and_unload() is not an in-place operation, so you must keep the returned model.

merged = model.merge_and_unload()   # returns a new model; assign it
merged.save_pretrained("ticket-labeler-merged")

Merging rules out serving many adapters on one base model at the same time, so keep the unmerged adapter if you need that. In production, log inputs and outputs under your privacy rules, re-run the holdout evaluation on a schedule, and plan to retrain when the base model changes, because an adapter is tied to the base it was trained on.

The Dataset Mistakes That Ruin Both

Most failed LoRA and QLoRA runs trace back to the data, not the hyperparameters.

Duplicates and leakage. Repeated examples push the model toward memorization. Lee and colleagues found that deduplicating training text cut memorized output roughly tenfold and that train-test overlap affected over 4 percent of validation sets in common datasets. That work covers pretraining-scale data, so apply it to your set by analogy: deduplicate, and make sure no holdout example, or near copy of one, appears in training.

Inconsistent labels. A model cannot beat the agreement of the people who labeled its data. OpenAI's best-practices guide gives an example where labelers agreed on only 70 percent of cases, which caps what a model can learn. Northcutt and colleagues found an average of at least 3.3 percent label errors in the test sets of ten common benchmarks. Audit a sample by hand and write down labeling rules before collecting more.

Personal data and secrets. Scrub emails, phone numbers, keys and internal identifiers before training, because the model may repeat what it saw. Removing them afterward is far harder than never including them.

Chat-template mismatch. Hugging Face's documentation warns that wrong control tokens make models perform much worse, and that adding extra special tokens on top of a template that already includes them is often duplicated and hurts performance. Apply the same template in training and inference, and make sure the EOS token matches it.

Training on the prompt. If loss covers the system and user text, the model spends capacity learning to imitate inputs. Use assistant-only or completion-only loss where your template supports it.

No holdout, no stopping rule. Without held-out data you cannot see overfitting, which OpenAI describes as memorizing the dataset instead of learning something generalizable. Track evaluation loss during training, keep checkpoints and pick the best one by the holdout score rather than by the last epoch.

Which Companies Offer Fine-Tuning Services? 5 Teams Worth Shortlisting

Some teams prefer to hand the pipeline above to a specialist. The firms below list LLM fine-tuning or custom model training among their services. Figures come from each company's own website and are self-reported, so confirm them directly before signing anything.

1. Dev Technosys

• Founded: 2010
• Team Size: 250+
• HQ: Jaipur, India
• ISO 27001:2022 certified, ISO 9001:2015 certified and CMMI Level 3 appraised
• Seven-stage LLM fine-tuning process: discovery and data assessment, base model selection, dataset preparation, fine-tuning and training, evaluation and safety testing, deployment and integration, and monitoring and retraining
• Core Services: LLM fine-tuning, supervised and instruction tuning, LoRA and QLoRA adaptation, preference tuning, evaluation and benchmarking, RAG and hybrid optimization

Dev Technosys leads this list because its process mirrors the pipeline above: it starts with a use case and data readiness audit, prepares the dataset, builds evaluation and safety testing in before deployment and stays on for monitoring and retraining. It also treats retrieval and tuning as options to combine and not as rivals. For a business that wants one provider to own the work from the first data audit to the retraining schedule, that continuity is the practical advantage.

Best For: Businesses with proprietary data and a narrow, repeated task that need a custom model built, tested and maintained.

2. Azumo

Azumo, based in San Francisco and founded in 2016 by its own account, lists supervised fine-tuning, RLHF and DPO, LoRA and QLoRA, quantization and distillation on its LLM fine-tuning page. It says it benchmarks tuned models against the base model, A/B tests before release and can deploy in private cloud, on-premises or air-gapped environments. The page states no headcount, and the company also sells nearshore staffing, so buyers should confirm who does the training work. Azumo suits regulated fintech and healthcare teams that need air-gapped hosting from a small US-time-zone team.

3. Resourcifi

Resourcifi, a Wilmington, Delaware company that says it was established in 2017, cites 200+ experts, which is a bench figure and not a confirmed headcount. Its custom LLM page describes supervised fine-tuning with LoRA or QLoRA, preference tuning with RLHF or DPO and distillation, along with regression evaluations in CI that include checks for catastrophic forgetting. It also describes cleaning, deduplicating and versioning training data and serving models on the client's cloud or a private cluster. The page names no security certifications. Resourcifi suits product teams that want supervised and preference tuning with regression tests wired into their CI pipeline.

4. Geniusee

Geniusee says it was founded in 2017 and lists offices in Austin, Warsaw, Stockholm and Kyiv, with 300+ people. Its large language model page names LoRA and QLoRA for adapting pre-trained models without retraining from scratch, sets success metrics such as accuracy, response quality and latency before deployment, and offers private cloud, on-premises or hybrid hosting. It describes "ISO-compliant processes" without naming a certificate, and its experience claims differ between pages. Geniusee suits mid-size European and US buyers that want LoRA or QLoRA adaptation with a choice of private-cloud or on-premises hosting.

5. Folio3

Folio3, which lists a Pleasanton, California address and cites 700+ team members and 20+ years of experience, names LoRA and QLoRA, supervised fine-tuning, RLHF and DPO on its LLM fine-tuning page. It describes task-specific evaluations covering accuracy, consistency, safety, latency and robustness, and the building of clean training, validation and evaluation datasets. No security certifications are named on the pages reviewed. Folio3 suits mid-size and large buyers that want a full method list from a large firm with North American offices.

Frequently Asked Questions

What is the difference between LoRA and QLoRA?

LoRA trains small adapter matrices while the base model stays frozen in its normal precision. QLoRA does the same but loads the frozen base model in 4-bit precision, which reduces memory enough to tune much larger models on a single GPU. QLoRA saves memory at the cost of some extra complexity in setup.

How many examples do I need to fine-tune a model?

There is no universal number. OpenAI's guide reports improvements from 50 to 100 well-crafted examples for simple tasks and a minimum of 10 for its own service. Start small, measure against a holdout set and add data only if the score improves.

Should I fine-tune or use retrieval?

Use retrieval for facts that change often and tuning for consistent behavior, such as format, tone and narrow classification. Many production systems combine the two, and either one should be justified by an evaluation against a prompt-only baseline.

Is a fine-tuned model safe by default?

No. Tuning can weaken a model's safety behavior, and training data can leak into outputs. Remove personal data before training, test safety after every job and load weights only from sources you trust.