EVAL #003: Fine-Tuning in 2026 - Axolotl vs Unsloth vs TRL vs LLaMA-Factory

EVAL #003: Fine-Tuning in 2026 — Axolotl vs Unsloth vs TRL vs LLaMA-Factory

---

You don't need to fine-tune.

Let me say that again: most of you reading this do not need to fine-tune a model. You need better prompts. Maybe RAG. Maybe an agent loop. Fine-tuning is the nuclear option — powerful, expensive, and often overkill.

But when you DO need it, the difference between picking the right framework and the wrong one is weeks of wasted compute and a model that somehow got worse. So let's talk about when you actually need fine-tuning, and which tool to reach for in March 2026.

The Decision Framework: Do You Actually Need Fine-Tuning?

Before you spin up a GPU instance, walk through this:

Use prompt engineering when:

• Your task can be described in natural language
• You have fewer than 50 examples
• Latency isn't a hard constraint
• You're still figuring out what you want the model to do

Use RAG when:

• You need the model to reference specific, changing documents
• Factual accuracy on proprietary data matters more than style
• Your knowledge base updates frequently
• You want attribution and traceability

Use fine-tuning when:

• You need a specific output format the model consistently botches
• You're optimizing for latency and want a smaller model that punches above its weight
• You have domain-specific language or behavior (medical, legal, code)
• You have 1,000+ high-quality examples and the budget for experimentation
• You need to distill a large model's capabilities into something you can self-host cheaply

The honest truth: the bar for "you need fine-tuning" keeps rising. Base models in 2026 are scary good. Qwen3, Llama 4, Gemma 3 — they handle tasks out of the box that required fine-tuning 18 months ago. But when you cross that threshold, the tooling matters enormously.

The Landscape at a Glance

Here's where the four major open-source fine-tuning frameworks stand in March 2026:

Framework       | GitHub Stars | Latest Release   | Best For                    | Learning Curve
----------------|-------------|------------------|-----------------------------|---------------
LLaMA-Factory   | 68.4K       | v0.9.4 (Dec '25) | GUI-first, broad model support | Low
Unsloth         | 53.9K       | Feb 2026         | Speed/VRAM optimization      | Low-Medium
TRL             | 17.6K       | v0.15.0 (Mar '26)| RLHF/GRPO, HF ecosystem     | Medium-High
Axolotl         | 11.4K       | v0.29.0 (Feb '26)| Config-driven, production    | Medium

Stars aren't everything, but they tell a story. LLaMA-Factory and Unsloth have captured the community's attention. TRL is the institutional pick. Axolotl is the quiet workhorse.

Framework-by-Framework Breakdown

LLaMA-Factory (68.4K stars)

What it is: The most popular fine-tuning framework by raw numbers. Chinese-originated, now globally adopted. Ships with a web UI (LlamaBoard) that lets you configure and launch training runs from a browser.

The good:

• Broadest model support — if a model exists on HuggingFace, LLaMA-Factory probably supports it
• Web UI is genuinely useful for exploration and prototyping
• Just migrated to uv for package management (finally)
• Added Megatron-LM training via MCoreAdapter for serious distributed workloads
• New OFT (Orthogonal Fine-Tuning) support alongside standard LoRA/QLoRA
• KTransformers backend support for efficient inference after training
• Rebranded to "LlamaFactory" — dropped the caps, kept the momentum

The bad:

• Documentation quality is inconsistent. Some pages are excellent, others feel auto-translated
• The web UI can be a crutch — when something breaks, debugging through a GUI layer adds friction
• Release cadence has slowed (v0.9.4 was December 2025, nothing since)
• Now requires Python 3.11+ which can conflict with some CUDA/driver combinations
• Config sprawl is real — too many knobs without clear guidance on which ones matter

The honest take: LLaMA-Factory is where most people start, and for good reason. It has the lowest barrier to entry and covers the widest surface area. But "covers" and "excels" are different things. If you need to do something specific — like GRPO training or maximum performance on a single GPU — the specialized tools do it better.

Unsloth (53.9K stars)

What it is: The speed demon. Built from the ground up around custom Triton kernels that make fine-tuning dramatically faster and more memory-efficient. Founded by Daniel and Michael Han, who have been relentless about performance optimization.

The good:

• 2x-5x faster training than stock HuggingFace on standard workloads, and now claiming 12x on MoE models
• 35% less VRAM usage means you can fine-tune models on hardware that shouldn't be able to handle them
• Excellent Colab/notebook experience — their notebooks just work
• February 2026 release added MoE training support (huge for DeepSeek-style models)
• Embedding model fine-tuning support is a nice differentiator
• Added ultra-long context support for RL training
• FP8 training support since late 2025
• Hit 50K GitHub stars — momentum is real

The bad:

• The speed gains come from custom kernels that can lag behind new model architectures
• When a new model drops, you might wait days/weeks for Unsloth support
• The "Pro" tier creates ambiguity about what's open source and what isn't
• Less flexibility for exotic training setups — it's optimized for the common case
• Multi-node training story is still developing compared to Axolotl or TRL

The honest take: If you're training on 1-2 GPUs and want maximum efficiency, Unsloth is probably your best bet. The speed improvements are real, not marketing. But it's an optimization layer, not a full training platform. For anything beyond standard SFT and LoRA, you'll need to combine it with other tools. The good news: Unsloth plays well with TRL and the broader HF ecosystem.

TRL — Transformer Reinforcement Learning (17.6K stars)

What it is: HuggingFace's official library for training language models with reinforcement learning. What started as a PPO implementation has grown into the canonical RLHF/alignment toolkit.

The good:

• GRPO (Group Relative Policy Optimization) implementation — this is what DeepSeek used, and TRL's version is the reference implementation
• Deeply integrated with the HuggingFace ecosystem (transformers, datasets, accelerate, peft)
• Battle-tested at scale by HF's own research team
• Clean, modular API that's become the standard other tools build on
• Active development — v0.15.0 just shipped in March 2026
• If you're doing alignment research, this is the tool

The bad:

• Not optimized for pure speed — you're paying a performance tax for generality
• The API changes frequently between versions — migration can be painful
• SFT capabilities are basic compared to Axolotl or LLaMA-Factory
• Documentation assumes you already understand RL concepts
• Debugging distributed training issues requires deep HF stack knowledge

The honest take: TRL is the framework you reach for when the training objective matters more than training speed. GRPO, DPO, PPO, RLOO — if you're doing anything beyond supervised fine-tuning, TRL is the answer. It's not the fastest and it's not the easiest, but it's the most correct. If your workflow is "I want to do RL on a language model," start here.

Axolotl (11.4K stars)

What it is: A config-driven fine-tuning framework that emphasizes reproducibility and production readiness. Originally from the OpenAccess-AI-Collective, now under Axolotl AI Cloud.

The good:

• YAML-driven configuration makes runs reproducible and shareable
• Excellent multi-GPU and multi-node support out of the box
• Strong DeepSpeed and FSDP integration
• Most complete data preprocessing pipeline — handles conversation formats, packing, etc.
• The framework of choice for many serious fine-tuning shops
• v0.28.0 and v0.29.0 shipped in quick succession (Feb 2026) — development is active

The bad:

• Lowest star count of the four — smaller community means fewer Stack Overflow answers
• Config files can get complex fast, and the documentation doesn't always keep pace
• Less focus on single-GPU optimization (that's Unsloth's territory)
• Initial setup is more involved than the alternatives
• Error messages could be more descriptive

The honest take: Axolotl is the framework I'd recommend if you're building a fine-tuning pipeline that needs to run reliably in production. The config-driven approach means your training runs are version-controllable and reproducible. The data preprocessing is best-in-class. It doesn't have the flashiest features or the most stars, but it's the tool that serious practitioners keep coming back to. Think of it as the framework you graduate to.

Recommendation Matrix

"I just want to try fine-tuning for the first time"
→ LLaMA-Factory. The web UI will get you running in minutes.

"I have one GPU and need to make it count"
→ Unsloth. Nothing else comes close on single-GPU efficiency.

"I need RLHF/GRPO/DPO alignment training"
→ TRL. Built for exactly this. Combine with Unsloth for speed.

"I'm building a production fine-tuning pipeline"
→ Axolotl. Config-driven, reproducible, battle-tested at scale.

"I'm training MoE models"
→ Unsloth (for speed on fewer GPUs) or LLaMA-Factory (for Megatron-LM scale).

"I'm fine-tuning vision-language models"
→ LLaMA-Factory has the broadest VLM support today.

"I need to fine-tune and I'm already deep in the HF ecosystem"
→ TRL + Unsloth. Use what you know, add speed where you can.

The meta-trend here: these tools are increasingly complementary, not competing. Unsloth's kernels work with TRL's trainers. Axolotl can leverage both. The lines are blurring, and that's good for everyone.

---

The Changelog

Notable releases from the past two weeks across the AI/ML ecosystem:

1. TRL v0.15.0 (Mar 6) — Latest from HuggingFace's RL training library. Continued GRPO refinements and documentation overhaul.

2. Unsloth February 2026 Release (Feb 10) — 12x faster MoE training, embedding model support, ultra-long context RL. The team crossed 50K GitHub stars.

3. Axolotl v0.29.0 (Feb 25) — Quick follow-up to v0.28.0 with stability improvements and new model support.

4. LLaMA-Factory v0.9.4 (Dec 31, 2025) — OFT support, Megatron-LM integration, KTransformers backend, migrated to uv. Rebranded from LLaMA-Factory to LlamaFactory.

5. DeepSeek-OCR 2 — New OCR model with broad fine-tuning framework support already landing.

6. Qwen3 support rolling out across Unsloth and LLaMA-Factory — the Alibaba model family continues to gain traction in the open-source community.

7. Astral uv adoption — LLaMA-Factory's migration to uv signals broader ecosystem movement away from pip for ML project management.

---

The Signal

1. GRPO is eating RLHF. Group Relative Policy Optimization — the technique DeepSeek popularized — is rapidly becoming the default alignment method. It's simpler than PPO (no critic model needed), more stable than DPO, and frameworks are racing to optimize their implementations. TRL has the reference implementation but Unsloth is adding speed-optimized versions. If you're still using vanilla DPO, you're already behind the curve.

2. The single-GPU fine-tuning era is peaking. Between QLoRA, Unsloth's kernel optimizations, and FP8 training, you can now fine-tune a 70B-parameter model on a single 24GB GPU. This was science fiction two years ago. The implication: the barrier to custom models has never been lower, which means the differentiator is shifting from "can you fine-tune" to "do you have the data and eval pipeline to fine-tune well."

3. Fine-tuning frameworks are converging on the same models and diverging on workflow. Every framework now supports Llama, Qwen, Gemma, DeepSeek, and Mistral within days of release. The competition has moved upstream to developer experience — GUI vs config files vs notebooks vs API. This is healthy. Pick the workflow that matches your team, not the one with the most stars.

---

That's EVAL #003. The fine-tuning landscape is maturing fast — the tools are better, the models are better, and the decision of when to fine-tune is getting clearer. The hard part was never the framework. It's the data.

If someone forwarded this to you — first, they have good taste. Second, subscribe so you don't miss the next one.

Subscribe: buttondown.com/ultradune
GitHub: github.com/softwealth/eval-report-skills

— EVAL