DEV Community: Sarthak Agrawal

PostTrainLLM vs Axolotl: Mac specialist workflow or accelerator framework?

Sarthak Agrawal — Fri, 04 Sep 2026 04:30:00 +0000

Axolotl is a mature configuration-driven framework for fine-tuning across supported accelerator environments. PostTrainLLM is an Apple Silicon-first workflow for taking a narrow specialist from target definition through MLX training, evaluation, routing, packaging, and a documented ship-or-reject decision.

Axolotl's own Mac documentation describes M-series support as partial, so hardware is the first branch in this comparison.

The practical comparison

Question	PostTrainLLM	Axolotl
Primary job	End-to-end Mac specialist factory and evidence workspace	Configuration-driven framework for post-training many model families
Apple Silicon	Core path through MLX and local runtime experiments	Official Mac documentation describes partial support
Distributed training	Not the product center	Advanced GPU and distributed configurations on supported environments
Configuration	Recipes plus explicit datasets, baselines, evaluations, reports, and routing decisions	YAML-centered training configuration with broad model and method options
Best fit	Reproducible specialist experiments on Apple Silicon	Flexible production training on supported Linux/GPU infrastructure

When Axolotl is the better fit

Axolotl is a natural choice when an organization already has a supported GPU environment and needs a flexible training framework.

Its documentation covers LoRA and QLoRA optimizations, multimodal models, preference and reinforcement-learning methods, and distributed stacks. A configuration-first workflow also helps when many experiments share one repeatable training shape.

The same feature matrix should not be assumed on a Mac. Axolotl's M-series page documents partial support and material constraints around PyTorch MPS, dependencies, training modes, precision, kernels, bitsandbytes, DeepSpeed, and related capabilities.

When PostTrainLLM is the better fit

PostTrainLLM is narrower in hardware posture and broader in experiment lifecycle.

It is built to answer whether a specialist should exist: define the target, freeze the baseline and holdout, prepare data with provenance, run a Mac-compatible recipe, evaluate target and breadth behavior, package the artifact, and record the route or rejection.

That does not make it a replacement for Axolotl's distributed-training breadth.

Do not compare configuration files alone

A YAML file can reproduce training parameters while leaving the product question unanswered.

A fair test needs a frozen model revision, tokenizer and chat template, dataset split, adapter targets, sequence length, precision, stopping rule, and evaluation suite. It should report target quality, breadth regressions, peak memory, wall time, artifact compatibility, and failure cases.

PostTrainLLM does not publish a controlled Axolotl head-to-head run, so I make no claim that its MLX path trains faster, uses less memory, or produces a better model.

The full comparison and current primary sources are at https://posttrainllm.com/posttrainllm-vs-axolotl.

PostTrainLLM vs Unsloth: start with the hardware and the decision

Sarthak Agrawal — Thu, 03 Sep 2026 04:30:00 +0000

PostTrainLLM and Unsloth are not interchangeable products.

Unsloth is a mature, optimized training and inference framework across supported accelerator hardware. PostTrainLLM is an Apple Silicon-first specialist-model workflow that connects data preparation, MLX recipes, evaluation, routing decisions, artifacts, and local runtime evidence.

The choice starts with the hardware and the outcome you need.

The practical comparison

Question	PostTrainLLM	Unsloth
Primary job	Mac-first factory for building and evaluating narrow specialists	Optimized open-source training and inference framework
Apple Silicon training	Core path through MLX and MLX-LM recipes	Official requirements currently describe MLX training as work in progress
NVIDIA training	Not the product center	Core strength across LoRA, QLoRA, full fine-tuning, and reinforcement-learning workflows
Evidence posture	Frozen baselines, target and breadth evaluations, report cards, artifacts, and limitations	Training metrics and framework benchmarks; the user owns the product-evaluation boundary
Best fit	Apple Silicon experiments where downstream evaluation matters	Fast training on supported hardware with broad model and method coverage

When Unsloth is the better fit

Unsloth is the stronger default when efficient training on officially supported hardware is the main problem.

Its documentation covers many models, LoRA and QLoRA, full fine-tuning, pre-training, reinforcement learning, notebooks, Studio, monitoring, and export. A team with NVIDIA infrastructure and an established dataset and evaluation stack should consider it before adopting a broader experimental factory.

Unsloth can run GGUF models and several Studio workflows on macOS. That is different from native Apple Silicon training, and its current requirements should be rechecked because MLX support is actively changing.

When PostTrainLLM is the better fit

PostTrainLLM is designed around the complete specialist experiment on a Mac.

It starts from a narrow target, freezes a baseline and holdout, prepares the data, runs an MLX-compatible recipe, measures target gain and breadth regression, packages the artifact, and records a ship, retry, route, or reject decision.

That is useful when the difficult question is not how to launch a fine-tuning command, but whether the specialist earned a narrow route and can be reproduced on Apple Silicon.

A fair evaluation plan

A valid comparison needs a shared model, dataset, prompt template, precision, adapter target, sequence length, batch policy, and frozen evaluation set.

I would record wall time, peak memory, adapter size, target accuracy, breadth regressions, and export/runtime compatibility. A Mac MLX run and a CUDA result should not be compared as though hardware and kernels were controlled.

PostTrainLLM does not publish a controlled Unsloth head-to-head result, so I make no speed, memory, or quality superiority claim.

The complete comparison and current primary sources are at https://posttrainllm.com/posttrainllm-vs-unsloth.

CodeVetter vs Greptile: repository-aware review or task evidence?

Sarthak Agrawal — Wed, 02 Sep 2026 04:30:00 +0000

Greptile is a repository-aware review product that works in the pull-request workflow and can hand findings to coding agents. CodeVetter is an execution-backed verifier that binds a requested task and exact change to checks, artifacts, and a measurable verdict.

The useful choice is not which product sounds more advanced. It is which authority the team needs: review guidance or reproducible task evidence.

The practical difference

Question	CodeVetter	Greptile
Primary job	Verify whether an agent task worked and retain evidence	Review pull requests with repository context
Main surface	Local viewer, CLI/MCP, and portable evidence	GitHub/GitLab review workflow, CLI, and agent handoff
Product center	Task identity, executable checks, and verdict portability	Continuous repository-aware review
Deployment direction	Local product without a required CodeVetter-hosted code service	Hosted cloud with enterprise self-hosting documented

When Greptile fits

Greptile is built around continuous pull-request review with repository context. Its official material describes full-codebase analysis, configurable rules, GitHub and GitLab support, comments that can be sent to coding agents, a CLI, and enterprise self-hosting.

Its TREX surface can also run code during review, so it would be inaccurate to call Greptile static-only.

When CodeVetter fits

CodeVetter makes the portable verification record the primary artifact. It begins with the task and exact patch, executes repository-owned checks, classifies failures, preserves bounded outputs and artifacts, and emits pass, fail, or unverified without turning missing evidence into confidence.

That is useful for agent evaluation, repeated regression comparison, and local or air-gapped evidence review.

How TREX changes the comparison

TREX brings Greptile closer to verification by adding execution. The remaining difference is the center of gravity and evidence contract: Greptile starts from managed pull-request review, while CodeVetter starts from immutable task identity, execution evidence, and a portable verdict.

Teams should test both against their own acceptance checks.

No invented leaderboard

CodeVetter's public benchmark compares its current review pipeline with a raw Claude baseline on 27 synthetic cases and 29 labeled findings. Greptile has not been run through that harness.

Greptile publishes separate benchmark material, but separately designed benchmarks cannot become a shared leaderboard. I therefore make no claim that either product catches more bugs, produces fewer false positives, runs faster, or costs less per successful verification.

The full comparison and current primary sources are at https://codevetter.com/codevetter-vs-greptile.

A small language model specialist should earn one job

Sarthak Agrawal — Tue, 01 Sep 2026 04:30:00 +0000

A small language model specialist does not need to outperform a frontier model everywhere. It needs to beat a chosen baseline on one defined target, remain inside regression and runtime limits, and route away when the task is outside its competence.

That is a more useful standard than calling every narrow improvement a smarter model.

Pick the job

Good specialist targets have repeatable inputs, objective outcomes, representative data, and a runtime reason to stay local.

Tool selection, structured extraction, file operations, SQL generation, formatting, and bounded intent routing are measurable. "Be a better assistant" does not define a measurable target.

Establish the baseline

I run the unmodified base model on the frozen evaluation set before training. The baseline should include accuracy by slice, invalid outputs, latency, memory, and abstention or routing behavior when available.

A specialist without a baseline has no demonstrated gain.

Choose the smallest intervention

Prompting or constrained decoding comes first when it solves the problem. LoRA or QLoRA makes sense when behavior has to be learned. Distillation can help when a stronger teacher produces reviewed target behavior.

I avoid stacking methods until the failure requires them.

Protect breadth with routing

A candidate can improve on its target and regress on general work. It needs an explicit router or confidence boundary, and that boundary needs its own evaluation.

The runtime should select the specialist only for tasks it has earned. A routed ship should be reported as routed, not as a new general default.

Package the whole result

A usable package includes the base identity, adapter or fused artifact, tokenizer and prompt contract, routing rule, runtime requirements, license information, evaluation report, known regressions, and reproducible configuration.

Missing latency or memory remains missing.

Make an honest decision

The outcomes I use are:

ship inside the declared route
retry training
retry the data or evaluation
reject

Failed attempts should remain available because they explain the next experiment and prevent the same recipe from being repeated without evidence.

PostTrainLLM is organized around this target-to-report loop. Its public file-operations specialist shows a strong target gain alongside a disclosed breadth regression and routing constraint.

The complete specialist workflow is at https://posttrainllm.com/build-small-language-model-specialist.

CodeVetter vs CodeRabbit: review guidance or verification evidence?

Sarthak Agrawal — Mon, 31 Aug 2026 04:30:00 +0000

CodeRabbit and CodeVetter address different checkpoints.

CodeRabbit is built for automated review inside pull requests, an IDE, or a CLI. CodeVetter starts from an agent task and exact change, then binds them to executable checks, retained evidence, and a pass, fail, or unverified verdict.

A team can use both: review to discover risk, verification to prove behavior.

The practical difference

Question	CodeVetter	CodeRabbit
Primary job	Execution-backed verification of an agent change	Automated AI code review and development workflow
Main surface	Local desktop viewer plus CLI/MCP and portable evidence bundles	Pull requests, IDE, CLI, and planning workflow
Main output	Task-linked checks, artifacts, uncertainty, and measurable verdict	Review summaries, findings, suggestions, fixes, and conversations
Product center	Reproducible behavior and evidence	Team review workflow and developer feedback

When CodeRabbit is the natural fit

CodeRabbit meets developers where review already happens. Its official documentation covers automatic pull-request reviews, incremental follow-ups, IDE and CLI review, codebase knowledge, linked repositories, planning, and one-click fixes.

If the immediate need is consistent review comments across a team and Git provider, that is its natural workflow.

When CodeVetter is the natural fit

CodeVetter starts from the requested task rather than the review comment. It records the exact change, runs declared checks in a bounded environment, separates regressions from environment and pre-existing failures, and preserves a machine-readable verdict.

That distinction matters when a plausible patch must demonstrate browser behavior, an API contract, authorization, persistence, concurrency, or regression closure.

Do not invent a head-to-head result

CodeVetter publishes 27 synthetic cases with 29 labeled findings, outputs, scoring rules, and limitations. CodeRabbit has not been run through the same harness.

That means I do not claim that either product catches more bugs, produces fewer false positives, runs faster, or costs less on the basis of CodeVetter's benchmark. A valid comparison would require the same immutable cases, configuration disclosure, repeated runs, and scorer.

A combined workflow

Let CodeRabbit identify suspicious code in the pull request.
Translate material findings and acceptance criteria into focused executable checks.
Run CodeVetter against the exact agent change.
Keep review findings and execution evidence separate.
Re-run failed behavior after the fix and preserve both attempts.

The full comparison, including current source links and data-boundary notes, is at https://codevetter.com/codevetter-vs-coderabbit.

An agent trajectory dataset needs more than final answers

Sarthak Agrawal — Sun, 30 Aug 2026 04:30:00 +0000

An agent trajectory dataset should preserve the behavior that produced an outcome, including the steps before the final response.

That means keeping the task, observations, tool calls, tool results, decisions, corrections, and authoritative outcome. Without provenance, privacy controls, and a frozen evaluation boundary, a large trajectory collection can still be a poor training dataset.

Choose the training unit

One example might be a full episode, a single decision point, or a correction pair. I keep the original task and enough state to explain each action.

Hidden evaluator feedback that the runtime will never receive should not leak into the training input.

Capture typed events

Observations, permitted reasoning, tool names, arguments, results, errors, and final outputs should use a versioned schema.

Raw source references need to remain available so the transformation can be audited. Flattening every event into one undifferentiated chat transcript throws away useful structure.

Filter for useful behavior

I remove duplicates, leaked tests, private values, malformed tool calls, and trajectories without an authoritative outcome.

Successful examples, correction data, and hard negatives should stay distinguishable. Every acceptance or rejection should retain its reason.

Split by source and task family

Random row splits can place nearly identical episodes in training and test data. I group by repository, task template, session, or source artifact before splitting and freeze the holdout before tuning the recipe.

Train the narrowest useful target

Trajectory data can support supervised fine-tuning, preference pairs, distillation, routing, or an evaluator. The objective should follow the observed failure.

If the model already over-edits, an objective that increases edit pressure is the wrong response.

Evaluate behavior, not imitation

The important measures are whether the specialist chooses the right tool, produces valid arguments, responds correctly to results, stops at the right point, and completes the task.

Unsupported actions and breadth regressions need their own gates.

PostTrainLLM includes trajectory conversion, correction-to-data, synthesis, filtering, deduplication, tool-calling evaluations, and factory-run evidence. The full workflow is at https://posttrainllm.com/agent-trajectory-dataset.

Meeting notes can stay on the Mac

Sarthak Agrawal — Sat, 29 Aug 2026 04:30:00 +0000

Meeting-note software often begins with a convenient transcript and quietly turns the entire conversation into a third party's input.

For customer data, hiring discussions, product strategy, credentials, or other sensitive work, I want a different default: capture, transcription, synthesis, and retention on the Mac.

The complete local pipeline

A useful meeting workflow needs more than a microphone recording. It should:

capture the user's microphone and the other side of the call
preserve the tracks
transcribe them locally
segment the conversation
produce a summary, decisions, and action items
link important output back to source moments
expose storage, retention, export, and deletion

The last two steps are easy to omit and are essential when generated notes need to be checked.

Pace meeting mode

Pace records two local audio tracks: the user's microphone and system or call audio. When the meeting ends, local transcription feeds a privacy-pinned local note model.

The resulting notes include structured sections and transcript-grounded action items. Meeting files remain ordinary local artifacts under Pace's application-support directory and follow a configurable retention policy. The current public FAQ describes a 30-day default.

Why the privacy pin matters

Pace can use a cloud or command-line planner for other capabilities when the user explicitly selects it. That choice does not allow meeting-note synthesis to leave the local path.

The meeting pipeline has its own stricter boundary. Users can still export content or invoke another external workflow themselves, but the default synthesis remains local.

What I would evaluate

Does the app capture both sides reliably on the target Mac?
Do transcription and synthesis work without a network fallback?
Can an action item show its source quote or timestamp?
Are files easy to locate, export, and delete?
Is retention configurable and enforced?
Are microphone and system-audio permissions explained clearly?

Local does not mean unlimited

Local transcription and synthesis consume memory, storage, and battery. Accuracy still depends on audio quality, speakers, language, and installed models.

A generated summary is a draft. Important decisions should be checked against the transcript rather than accepted because the model sounds certain.

The full meeting pipeline and current limitations are at https://heypace.app/on-device-meeting-notes-mac/.

What CodeVetter's public benchmark proves, and what it does not

Sarthak Agrawal — Fri, 28 Aug 2026 04:30:00 +0000

CodeVetter's public v1 benchmark is a reproducible recognition benchmark for agent-written bugs. It publishes 27 synthetic cases, 29 labeled findings, reviewer outputs, scoring rules, downloads, and explicit limitations.

That answers a narrow question: does the tested review pipeline recognize these known issues under this fixture set? It does not prove that CodeVetter verifies arbitrary production pull requests.

Read the result in layers

I separate the evidence on the benchmark page into three layers:

Published now: the 27-case recognition benchmark and its exact results.
Implemented qualification infrastructure: task packages, executable checks, receipts, and evaluation contracts described by the repository.
Not yet proven publicly: repeated provider runs on a broad realistic corpus with reliability, cost, and latency comparisons.

Blurring those layers would turn a useful result into a marketing claim it cannot support.

Choose the benchmark for the question

Recognition fixtures help isolate issue-type coverage and scoring behavior.

Repository-task benchmarks such as SWE-bench ask whether a generated patch resolves a real issue inside a reproducible environment.

A product-specific agent benchmark should go further by retaining the requested task, exact patch, environment, checks, evidence, and uncertainty. No single score answers all three questions.

Inspect the artifacts

A benchmark should expose its dataset, per-case pages, reviewer outputs, scoring method, and reproduction path. It should also put the limitations before the call to action.

For this benchmark, those limits include synthetic cases, narrow languages and categories, no timing or cost dimension, and precision as a material constraint.

The next credible proof

The next step is not a larger slogan. It is qualified, repeated agent-task evidence with immutable receipt linkage and a clear failure taxonomy.

You can inspect the cases, downloads, scorer, and current limitations at https://codevetter.com/benchmark.

Evaluate a local LLM for a decision, not a leaderboard

Sarthak Agrawal — Thu, 27 Aug 2026 04:30:00 +0000

A local LLM evaluation should answer a product decision. Producing one more leaderboard number is not enough.

I freeze the target and baseline first, run the unchanged model and candidate through the same protocol, inspect target slices and regressions, and then record one of four outcomes: ship, route, retry, or reject.

Define the contract

The evaluation contract needs inputs, expected outputs, allowed variation, protected content, and failure classes.

Metrics should match the task. SQL needs execution correctness. Tool agents need tool and argument accuracy. Autocorrection needs unnecessary-edit and meaning-preservation checks. Retrieval needs grounded-answer criteria.

Qualitative review still matters, but it should not overwrite deterministic gates.

Freeze the evaluation set

Training, validation, and test data need separate provenance. I deduplicate across splits and freeze the holdout before tuning the recipe.

A holdout that changes for every candidate cannot support a fair comparison. Synthetic fixtures should stay labeled as synthetic, and a small suite should never be described as production proof.

Run baseline and candidate symmetrically

Both models need the same prompts, decoding settings, runtime, and evaluator. Raw predictions and measurement state should be preserved.

Timeouts and missing values are different from zero. When the candidate is a routed specialist, the target slice and the routing boundary both need evaluation.

Measure regressions directly

A specialist can improve its hard gate and become worse elsewhere. I report the delta for every protected slice and set reject conditions before the run.

Those conditions can include meaning changes, leakage, excessive breadth loss, or latency and memory outside the target machine's budget.

Make the decision reproducible

The final package should identify the base and candidate, dataset revision, evaluator revision, configuration, per-slice results, missing measurements, and decision.

"Ship" must remain narrower than "the candidate improved." A routed ship can be correct when the target gain is real and the route is enforced.

PostTrainLLM includes fixture and no-GPU gates, live evaluation commands, canonical factory-run records, packages, and public report cards. Missing historical latency, RAM, throughput, or raw traces remain missing instead of being replaced with estimates.

The full evaluation workflow is at https://posttrainllm.com/evaluate-local-llm.

A Mac AI assistant should verify its actions

Sarthak Agrawal — Wed, 26 Aug 2026 04:30:00 +0000

A useful Mac assistant needs to do more than generate instructions. It also needs to do less than silently control the computer.

The boundary I want is an observable action loop: understand the request, inspect the current state, propose typed steps, request approval when risk requires it, act through the strongest available interface, and verify what changed.

Prefer semantic actions

Application APIs, App Intents, Shortcuts, MCP tools, and the macOS Accessibility API expose meaning and state. They are usually more durable than raw screen coordinates.

Visual grounding and pointer events still matter for applications with incomplete semantic access, but I treat them as a fallback. A coordinate can tell the system where input was sent. It cannot prove what the control meant or whether the outcome was correct.

Keep planning separate from authority

A model can propose an action. Policy should decide whether it may run.

Read-only navigation, reversible edits, external messages, downloads, and destructive mutations require different treatment. Planner output should become typed actions that can be validated before execution, not free-form instructions with ambient authority.

Verify every material step

A click event proves only that input was sent. Verification should inspect the resulting window, control value, file, reminder, message draft, or other expected state.

If the result is ambiguous, the assistant should stop or recover. It should not continue a long workflow on an assumed success.

Pace's current action stack

Pace combines a typed tool registry, a plan-act-observe loop, application integrations, App Intents and deeplinks, accessibility and pointer execution, click scoring and recovery, undo for supported reversible mutations, failure narration, and local action history.

Users can also record literal flows, install recipes, or teach natural-language skills that are re-grounded against the current screen. MCP connections and external services remain optional.

Start with bounded workflows

The best early workflows have outcomes that are easy to inspect:

open an application
create a reminder
draft rather than send an email
navigate a known interface
record a repeatable flow
guide the user through an unfamiliar screen

Authority can expand after the verification path proves reliable.

The current action and privacy model is at https://heypace.app/mac-ai-assistant-actions/.

What belongs in a coding-agent verification evidence bundle?

Sarthak Agrawal — Tue, 25 Aug 2026 04:30:00 +0000

A verification evidence bundle is the portable record behind a verdict. It should let a person, a CI system, or a later evaluation understand what was checked without depending on a screenshot or a model summary.

When I cannot answer which task, revision, environment, and checks produced a result, I do not consider that result durable evidence.

Bind the required identities

The bundle should identify the task, repository, base revision, agent change, verifier version, environment, and attempt.

Without those identities, passing output can be attached to the wrong patch or repeated under different conditions. The result may be real and still fail to prove the change under review.

Preserve execution records

For every check, I want the normalized command or runner identity, start and finish state, exit result, timeout or cancellation state, and bounded output.

Screenshots, reports, and other artifacts should be recorded by path and digest when they contribute to the verdict. Environment failures should remain environment failures rather than being hidden inside a generic test failure.

Keep uncertainty in the verdict

A useful verdict separates:

passed requirements
reproduced failures
regressions
pre-existing failures
infrastructure failures
requirements that remain unverified

If an authoritative check is missing, the bundle should fail closed. A confidence score cannot fill an evidence gap.

Redact without destroying portability

Evidence needs to be useful without leaking secrets or entire private repositories. I retain the necessary excerpts and metadata, enforce output bounds, redact sensitive values, and make truncation explicit.

JSON works well for automation. Markdown and self-contained HTML are useful when a person needs to inspect the result offline.

Link the re-check

When a failure is fixed, the new bundle should reference the earlier attempt and preserve the failing evidence. That gives the work a closure trail:

failed behavior -> corrective change -> passing check -> regression result

CodeVetter treats this machine-readable bundle as a primary product surface. Completed reviews can export fail-closed JSON, Markdown, and self-contained HTML while keeping missing proof visible.

The full bundle model is at https://codevetter.com/verification-evidence-bundle.

MLX LoRA fine-tuning: the command is the easy part

Sarthak Agrawal — Mon, 24 Aug 2026 04:30:00 +0000

MLX LM gives Apple Silicon a native path for adapting open language models with LoRA or QLoRA. Starting the run is straightforward. Deciding whether the resulting adapter is useful is the real work.

I treat the dataset contract, frozen baseline, and post-training evaluation as part of the training recipe rather than cleanup after it.

Prepare the environment and data

MLX LM accepts a local data directory with train.jsonl, optional valid.jsonl, and test.jsonl, as well as supported Hugging Face datasets. Chat, tool, completion, and text formats can all work.

Each JSONL example should stay on one line. The selected base model's chat template needs to be checked before a long run. A malformed template can turn a valid-looking dataset into the wrong learning problem.

Run a bounded adapter experiment

I use mlx_lm.lora with an explicit base model, data path, iteration budget, and adapter output path. A quantized base selects QLoRA; a non-quantized base uses LoRA by default.

The run record should keep the exact base revision, MLX LM version, configuration, seed, data manifest, and machine. If prompt masking is enabled, I verify that loss is applied to the intended completion tokens.

Evaluate before fusing

The unchanged base model and the adapter should run against the same frozen examples.

Perplexity can reveal some training problems, but it cannot replace task metrics. A tool specialist needs exact tool and argument checks. A formatter needs protected-span and unnecessary-edit measurements. Every narrow target also needs out-of-domain slices so a gain does not hide a breadth regression.

Generate and package

Qualitative generation is useful after the quantitative gate, not instead of it. I fuse an adapter only after the evidence supports that decision.

The package still needs the base identity, adapter or fused artifact, tokenizer, quantization, runtime requirements, model card, license checks, evaluation revision, and missing measurements.

Failure modes I watch for

overlapping training and evaluation examples
choosing a threshold after seeing the candidate
treating lower loss as task success
publishing only the improved slice
fusing before preserving the adapter, recipe, and evidence
replacing a general model with a narrow specialist

PostTrainLLM wraps this into a factory-run, evaluation, package, and report-card workflow. The complete MLX path is at https://posttrainllm.com/mlx-lora-fine-tuning.