Flash Onyx 2.4: I held a 31B local model to Fable 5.1's bar until it stopped guessing the answer first

Flash Onyx 2.4 is on Ollama.

Same gemma4 weights as 2.3. Same ten PARAMETER lines, down to repeat_last_n 256. The Modelfile went from 676 lines to 668 and gained 324 words doing it, which tells you what kind of release this is: 81 lines removed, 73 added, and nothing underneath them touched.

I did not train anything. I spent weeks fighting a text file, and the thing that came out the other side is the best local model I have ever run.

The rule that took the longest to find

Here is 2.3's speed rule, which I was proud of at the time:

Answer first. The verdict, the number, the command, or auth.py:88 goes in the opening words. The why comes after, if still needed.

It is a good rule. It is also how I taught my own model to lie to me.

Ask 2.3 what 15% of 8,240 is, and "answer first" is an instruction to put a number in the first four words. It does not have the number yet. So it produces one that looks like the right shape, then reasons underneath it, and about a third of the time the reasoning lands somewhere else. You get a reply that opens with 1,240 and closes with 1,236, and both of those are worse than either one alone, because now you have to do the arithmetic yourself to find out which half of the sentence to trust.

2.4 splits the rule in two:

That order is for what you already hold. A number or verdict you still have to compute or reason out never opens the reply: the short steps go first, the last step checks the result by a different route, and the answer comes last, once, because a figure stated before it was computed is a guess, and a reply that opens with one number and works out another is worse than either.

Answer first still holds for things it read, ran, or was told. For things it has to work out, the order inverts. Steps, check, answer, once.

The check is the part I would steal if I were you. Not "verify your work", which every model nods at and none of them do. A different route:

Try to break your own answer once, by a route other than the one that produced it: substitute it back, trace the code with concrete values, recount, test it against the constraint it cannot break (a part that takes five minutes to make is never made in less, however many machines there are).

That parenthetical is doing real work. Five machines make five widgets in five minutes; how long for a hundred machines to make a hundred widgets? Every model that pattern-matches on the numbers says a hundred minutes. The constraint that cannot break is that one widget takes five minutes, so a hundred machines still take five, and the extra machines sit idle if you ask for fewer widgets than you have machines. 2.4 has that written down as a class of problem, not as a fact about widgets.

It stopped claiming tool calls it never made

This one made me put the laptop down for a bit.

Ask a model with no tools attached whether ripgrep has some flag, and you will get "I checked, and there's no such flag." It did not check. There was nothing to check with. The sentence names a source that does not exist, which is the worst failure mode available to a model, because a fabricated source reads exactly like a real one and it comes with the confidence of the real thing attached.

2.4:

"I ran", "I checked", "the help output shows": each claims a tool call happened this session, and with no tool behind it that is a fabricated source, the worst kind. The honest form is "from memory, ripgrep has no such flag; rg --help | grep frob would settle it": what you recall, labeled, and the command that checks it.

Same idea reaches the ledger, which is the format 2.3 uses to report on anything with more than one part:

A DONE line carries the thing that proves it, quoted: the test output, the log line, the grep match. DONE with nothing quoted after it is the shape hallucinated progress takes.

I wrote a whole post once called "Your AI agent didn't finish. It just told you it did." This is the line that finally does something about it. DONE is now a claim that has to carry its receipt in the same line, and a model that cannot produce the receipt writes a different word.

Law, because it is where a made-up number gets someone hurt

I tested this thing on contract review and statutory questions for the same reason I tested it on pygame: if the rules only work on code, they are not rules, they are style.

2.3 already refused to invent citations. 2.4 closes the two gaps I kept falling into.

The first is that a model will quote a statute correctly and then state a damages figure next to it from vibes. So:

A damages figure is arithmetic done on the page, 2 x $2,000 = $4,000, never a number recalled next to the rule.

The second is subtler and it is the one I would want if I were relying on this. Recalled law is stale law. Statutes get amended, thresholds get renumbered, and a model's memory of a section number is a snapshot of whenever its data stopped:

Every section number, deadline, or dollar threshold you state from memory gets one clause saying so and naming where to confirm it.

Not a disclaimer paragraph at the bottom that nobody reads. One clause, attached to the specific number that came from memory, while you are looking at the number.

Games, where "it runs" and "it works" are different claims

The GAMES section is thirteen lines and it is my favorite part of the prompt. Fixed timestep with a clamped accumulator so a backgrounded tab does not spiral. Input polled, not handled. 100ms of coyote time and 150ms of jump buffering, because fairness is small lies. Nothing allocates in the loop, so pool the bullets and reuse the vectors, because a garbage pause reads as a stutter and players will tell you the controls feel bad without knowing why.

2.4's change is one clause in the final summary, and it is embarrassing how much it mattered:

A game is playable before it is pretty, on a fixed timestep, with a loss screen and a restart, and feel beats content.

2.3 built me a lot of beautiful things you could not lose. The loop was there, the physics were right, the particles were good, and the run never ended, so it was a toy. Nothing to lose, nothing to restart was already in the long section as "no win, no loss, no restart is a demo", but it was not in the compressed rules at the bottom, which is the part that survives a long context. Putting it there fixed it.

Same for code generally:

Everything you write has to run, complete: every import, every helper it calls, the entry point, and the command that runs it. No placeholders, no "for now", no scaffold with a comment describing what it should have been, no function left for the reader to fill in.

Plus a line that catches most of the rest: syntax-check it, trace it once with a concrete input, read it back end to end.

The filler pass, round four

Every Onyx release has one of these and I keep finding more. This round:

No acknowledgement openers. "Perfect!", "Great!", "Got it!", "Absolutely", and the worst one, which is any of those directly after a tool call, where the result is already on the screen and the model is congratulating it.

No open offers of help. "Happy to do that if you want" and "just say the word" join the ban list, with the reason attached, because reasons are what make a rule survive paraphrase: an open offer hands the work back and asks to be thanked for standing by.

No scare-quoting the user. This one I had never seen named anywhere. If you ask a model to make you an account and there is no account to make, it writes that it cannot "make an account", holding your own phrase at arm's length like the wording was the problem. You were describing a goal the ordinary way. Say the plain thing: there is no account to make.

No ** in a chat reply. Not as a heading, not for emphasis, not as the label on a list item. 1. Gravity multiplier: more of it while falling. is a list item. The same line with the label bolded is a report nobody asked for.

Math stays plain text, 7^222 and 3/4 and 25!, because a terminal renders LaTeX as raw dollar signs and I was tired of looking at them.

I also took robust and seamless off the kill-on-sight list. They are real words, they were in there because they show up in slop, and banning them was making the model reach for worse ones. The tell was never the vocabulary.

The tell was shape

This is the part I did not expect to have to write.

You can ban every LLM word and the output still reads as generated, because the giveaway is above the sentence:

Three bullets of matching length and matching grammar, or four paragraphs that all run four lines, read as generated even when every word in them is right. Real writing is lopsided: one item runs long because it had more to say, the next is three words. Let what you found set the shape, never a template you fill.

And the version of that which only shows up in a long session:

The machine shows up in the tenth reply, opening the way the last four opened and running the same four-line shape whatever was asked. Nobody has one greeting: before you open a turn the way you opened the last one, open it a different way.

2.3 varied its wording inside a reply. It did not vary across turns, so a twenty-turn session drifted into a rhythm and the rhythm was the tell. This is the single change that most made 2.4 feel like something else.

One more in the same family, cheap and worth it everywhere: call things what they call them. Their "export script" does not become "the data pipeline module" in the reply, because renaming their thing into your vocabulary makes them translate it back on every line.

The bar

I did not grade this against 2.3. Grading a prompt against the prompt it came from tells you it changed, not that it got better.

I did not grade it myself either. Claude Fable 5.1 graded it, on the same prompts, against the standard Fable 5.1 itself or GPT 6 Astra would be held to. Not "good for a local model." Good, or fix it. That is a harsh bar for a 31B open-weight model and it is the only bar worth having, because a frontier model marking your work does not grade on a curve it does not know about. Every rule quoted above exists because a run came back, Fable marked it down, and I went and found the line that caused it.

The honest framing, since 2.4 is a release about not overclaiming: that is one frontier model's judgment on my prompts, across law questions, games, and general coding, and a model grading a model is a judgment, not a measurement. I am not publishing a score, because I did not run a scored benchmark, and stating a figure I did not compute is the exact thing this release fixed. What I will say is that the gap that used to make me switch models mid-task closed, and it closed with English, not weights.

Run it

ollama run natuworkguy/flash-onyx-2.4:12b

Tag	Built on	Runs on
12b	gemma4:12b	your machine, consumer hardware
31b	gemma4:31b	your machine, and it wants a real GPU
31b-cloudbase	gemma4:31b-cloud	Ollama's cloud, no local weights

Nothing leaves the machine on the local tags.

In the Flash CLI, which is what the prompt is written for:

/model natuworkguy/flash-onyx-2.4:12b

2.3 is not retired. Same repo, same base, same tags, still builds:

python3 models/build.py models/flash-onyx-2.3.Modelfile

Identical weights and sampling between them, so if you want to see what a prompt is worth, run the two against the same eval with the same seed. That is the experiment I keep coming back to, and it keeps giving me a bigger number than it should.

The Modelfile is MIT, in models/ in the repo. Take the rules. The weights are Apache 2.0 and those terms are Google's, not mine: ollama show --license gemma4.

So: what is the one line in your system prompt that pulls the most weight? Mine, this round, was thirteen words about not saying the number until you have it.