I killed my fine-tune before I wrote a single line of training code

Everyone wants to say the words. "I fine-tuned my own model."

I almost did. Then I remembered where half my users actually run the thing.

The plan

Flash is a local AI shell. You talk to an Ollama model in your terminal and it runs commands, reads images, screenshots pages, clicks buttons.

It ships with Flash Onyx: custom models built for it. Onyx 2 sits on gemma4, 12b and 31b. Onyx 1 was llama3.1.

Right now, Onyx is a Modelfile. A system prompt, a stop set, num_ctx pinned to 65536, tuned sampling. That's it.

The next step was obvious: collect the tool-call traces, build a LoRA, merge, quantize to GGUF, ship flash-onyx-3.

What Cloudbase is

Quick definition, because it's my own word and it's the whole problem.

Ollama publishes some models twice. There's the tag you pull onto your own machine, and there's a -cloud tag that runs on Ollama's servers instead of yours. Same model, someone else's GPU.

Flash builds on both. When a Modelfile declares # cloud-base: true, build.py builds each size a second time against the base's -cloud tag and publishes it under my own suffix:

gemma4:31b         ->  flash-onyx-2.5:31b            (local weights)
gemma4:31b-cloud   ->  flash-onyx-2.5:31b-cloudbase  (Ollama's weights)

That -cloudbase tag is what people with a laptop instead of a 24GB card actually run. It's a real chunk of my users. And it can't end in -cloud itself, because Ollama reads that suffix as "go resolve this name on ollama.com," which mine is not.

The wall

Here is what a cloudbase build actually is: a system prompt, a stop set, and sampling params pointed at weights I do not host, cannot touch, and did not upload.

That's the entire surface. Ollama Cloud serves the base. I get to wrap it.

Which means there is no step in that pipeline where I hand anyone a file. I can't ship GGUF weights to a machine I don't own, and there's no hook to attach a LoRA adapter to a hosted base at load time. Not "slow." Not "expensive." There is no upload.

The pipeline just stops:

dataset -> LoRA -> merge -> GGUF -> ??? -> :31b-cloudbase
                                     ^
                           no door on this side

I could fine-tune for local users only. Then Onyx 3 is two different models wearing one name, the cloudbase half silently stays on v2 forever, and every bug report opens with "which tag are you on." No thanks.

The part nobody posts about

Fine-tuning content is all notebooks and loss curves. Almost none of it is "where does the weight file go and who serves it."

That question is upstream of everything. Answer it first:

• Can every runtime I ship to load a GGUF? (local: yes. cloudbase: no)
• Can any of them attach a LoRA at load time? (neither)
• If only some can, am I now maintaining two models under one name? (yes)
• Who eats cold-start and idle cost on a 24GB merge? (me, on the half that would even take it)

If any of those is a shrug, your fine-tune is a hobby, not a feature.

What I did instead

Kept iterating the Modelfile.

Turns out a sharp system prompt, honest stop tokens, and the right context window get you shockingly far on a base model that's already good at instructions. Onyx 2.5 is version five of a text file. Each one took an afternoon, not a GPU-week.

python3 models/build.py models/flash-onyx-2.Modelfile --size 12b

One text file, one build command, and both tags come out the far end running the same model. That's the part the fine-tune would have broken.

The takeaway

Your infrastructure sets your roadmap. Not your ambition.

Not fine-tuning isn't a failure. Shipping a fine-tune you can't deploy is.

Onyx 3 happens if Ollama stands up a 31b-cloud base on their servers and gives me a way to hand it a weight file. Until then: better prompts, same weights, one model everywhere.

---

Flash is open source: github.com/Natuworkguy/Flash

What would you do here? Fork the model and maintain two, hold the line on one, or something I haven't thought about? 👇