Launching a LoRA run on Apple Silicon is no longer the difficult part. The difficult part is deciding whether the resulting model is better, what it became worse at, and whether it should ship at all.
I built PostTrainLLM around that decision instead of treating training loss as the finish line.
Define the specialist before choosing a trainer
"Make the model smarter" is not an evaluation target.
A useful target describes the task, input and output contract, baseline model, acceptance slices, routing boundary, and failure conditions. File operations, SQL hygiene, and intent routing can be evaluated. A vague request for a better general chatbot cannot.
Freeze the holdout before generating or cleaning the training data. If the final test examples influence the training set, the resulting score cannot support a shipping decision.
Pick a method that fits the machine
LoRA changes a small set of adapter weights while keeping the base model frozen. QLoRA applies low-rank adaptation to a quantized base and can reduce memory pressure further. Full-weight fine-tuning can work for smaller models, but it is not my default starting point for a one-machine workflow.
Apple's MLX LM supports text generation, quantization, LoRA, QLoRA, and full fine-tuning on Apple Silicon. The method still has to fit the Mac's memory and the intended runtime. A candidate that cannot run inside its latency and memory budget has not passed.
Keep the factory loop connected
My working loop has six parts:
- Define the target behavior and routing boundary.
- Create train, validation, and frozen test splits with provenance.
- Run a bounded post-training recipe and preserve its configuration.
- Compare the unchanged baseline and candidate on the same gates.
- Package the adapter or fused model with runtime metadata.
- Report gains, regressions, missing measurements, and the ship, retry, or reject decision.
The point is traceability. A report card should identify the data revision, configuration, baseline, candidate, evaluator, runtime requirements, and known limits. Otherwise, reproducing the result becomes guesswork.
Do not hide the regression behind the win
One public PostTrainLLM artifact is a file-operations specialist. It improves its hard gate from 58% to 100%. Its out-of-domain breadth falls from 59.6% to 42.3%.
The first number looks like a success. The second changes how the model should be used. The candidate is routed as a specialist instead of being presented as a better general planner.
This is why I keep separate gates for target quality, regressions, latency, memory, and package integrity. An average score can hide a failure that matters to the product.
Package the evidence with the model
A shippable candidate should include more than weights. It needs the base model, adapter or fused artifact, tokenizer, quantization, runtime requirements, evaluation revision, limitations, and intended routing.
The report should also retain failed and missing measurements. A missing check is not a pass, and a narrow improvement is not a general capability gain.
A falling training loss can tell you that optimization is happening. It cannot tell you whether the model is safe to route, useful on the held-out task, or deployable on the target Mac.
The complete workflow is at https://posttrainllm.com/fine-tune-llm-on-mac. Public artifacts and report cards are available at https://posttrainllm.com/artifacts.
Top comments (1)
I like the separate regression gate here. The trap I keep seeing is teams treating a LoRA as a better model instead of a narrower route with a receipt stapled to it. Keeping the failed and missing measurements in the package is the part that makes the routing decision auditable later.