Okay, this is going to sound dumb, but the most expensive line item in one of my client pipelines last quarter was a model being asked to return JSON. Not to reason about anything. Just to take a support ticket, pull out six fields, and give them back as an object that json.loads wouldn't choke on. We were paying frontier-model prices for a task a regex would nearly do, because the cheap models kept returning a notes field nobody asked for, or wrapping the whole thing in a code fence when we said not to, or forgetting a required key one time in five.
So a Hugging Face post from last week got my attention. Leonie Monigatti, Ben Burtenshaw and Sergio Paniego take a 350M parameter model, LFM2.5-350M, run 100 steps of GRPO on about 500 samples, and lift its structured-output score from 22.6% to 29.7% on the IFStruct benchmark. On a free Colab GPU. In under an hour.
Those numbers aren't impressive on their own. 29.7% is still failing seven times out of ten. What's interesting is the shape of the recipe, because once you see it you realise the thing you're training isn't the model. It's the three reward functions. And those are the part you can write yourself.
What "structured output" is actually measuring
The IFStruct benchmark is worth understanding before the training bit makes sense. It's open source at Liquid4All/ifstruct, and it's narrowly about one thing: does the model return valid, parseable output in the format and shape you asked for? Not "is the answer right". Just "can this be wired into the next system without a human touching it".
The base model's failure breakdown on 2,000 test cases is where I'd start if I were you. 7,228 instances of a required field missing. 738 wrong item counts. 540 type mismatches. 317 unclosed code blocks. 190 cases of an extra notes field, 181 of an extra path, 175 of an extra constraints. The model isn't failing at understanding the schema. It's failing at discipline. It keeps adding helpful fields, keeps miscounting list items, keeps opening a fence and not closing it.
That's exactly the failure mode I was paying to avoid. And it's a much narrower problem than "make the model smarter", which is why a small model with a light nudge can move on it.
The recipe, minus the parts that don't matter
The post uses TRL's GRPO trainer. If you haven't touched GRPO, the short version: for each prompt, sample several completions (here, 8), score each one with reward functions, and push the model toward the completions that scored above the group average. No separate reward model, no preference pairs, no human labels. Just a scoring function that returns a number.
The LoRA config is nothing exotic, other than targeting LFM2.5's hybrid attention/convolution module names rather than the usual q_proj/v_proj pair:
lora_config = LoraConfig(
r=16,
lora_alpha=32,
task_type="CAUSAL_LM",
target_modules=["q_proj", "k_proj", "v_proj", "out_proj",
"in_proj", "w1", "w2", "w3"],
)
About 6M trainable parameters, 1.66% of the model. The training config is sized for a 16GB card:
training_args = GRPOConfig(
learning_rate=5e-5,
max_steps=100,
num_generations=8,
per_device_train_batch_size=4,
gradient_accumulation_steps=8,
max_completion_length=1024,
temperature=1.1,
beta=0.01,
reward_weights=[1.0, 0.5, 2.0],
)
Two of those numbers I'd flag. temperature=1.1 is hotter than you'd normally sample, and it's deliberate: GRPO needs the 8 completions in a group to actually differ, or there's no signal in "above average". And beta=0.01 is a small KL penalty toward the reference model, which is what stops 100 steps of "maximise the JSON score" from turning the model into something that only speaks JSON.
The rest of the pipeline is the TRL GRPO trainer doing what it does. I won't walk through it; the notebook is public and it runs.
The reward functions are the product
Here's the bit I keep coming back to. The three rewards, each on a 0 to 1 scale:
The first checks whether the output parses and is in the requested form. Fenced when asked for fenced, raw when asked for raw. Full credit for the right form, 0.2 for parseable but wrong form, zero for unparseable.
The second checks whether the top-level object has the expected number of fields, with the score decaying linearly as the count drifts.
The third validates the output against the row's JSON Schema, counts every violation, and gates partial credit on whether the required keys are present.
They're combined with weights of 1.0, 0.5 and 2.0. Schema validity is worth twice as much as parseability, which matches my experience of what actually breaks downstream.
Look at what's in there and what isn't. There's nothing about correctness. Nothing about whether the extracted invoice total is the right number. Nothing a human had to label. Every reward is a mechanical check you could write in an afternoon with json.loads and jsonschema.validate. Which means the "training data" for this is really just a pile of schemas and prompts, and the expensive part of fine-tuning, the labelled outputs, isn't needed at all.
If you've ever tried to build a preference dataset for something like this, that's the whole ballgame. I wrote about the moment a small fine-tuned model beat a frontier one on a narrow task and the labelling cost was the part that made it a hard sell to clients. Verifiable rewards remove that cost for any task where "did it comply" is a function you can write.
Where I'd push back
I don't want to oversell a 7-point lift. A few things bother me.
The gains are uneven and the post is upfront about the reason: the training data (NVIDIA's Nemotron structured-output set) doesn't look like the eval set, so the authors patch the gap by hand. 40% of prompts get a "return this in a fenced block" instruction appended, and a separate 20% are rewritten as bare-list tasks with a required item count. Those two augmentations map directly onto two of the base model's worst failure categories. That's smart, but it's also a reminder that the reward function isn't magic. You still have to know what the model gets wrong and put examples of it in front of the trainer.
The second thing: 29.7% is still a coin flip weighted against you. For a real pipeline I'd pair a model like this with strict decoding on the serving side, or with a retry loop that feeds validation errors back in. I covered why strict mode alone doesn't get you there with the big providers, and the same applies here in reverse. Constrained decoding guarantees the shape. It doesn't stop the model from filling the shape with nonsense. Training moves the model toward wanting to produce the right shape, which makes the constrained version's job easier and the failure rate on the unconstrained cases lower. You want both.
Third, and this is the honest one: the eval was run through llama.cpp on a MacBook, at BF16, with a specific --alias and context setting. The authors got 22.6% on the base model where the IFStruct paper reported 21.1%. That's close enough, but it tells you the number moves with the serving stack. Don't quote benchmark deltas across different servers.
What I'd actually do with this
For the support-ticket pipeline I opened with, here's the plan I've sketched and will try this month. Take the 40 or so ticket schemas we use in production. Generate a few hundred synthetic tickets per schema with whatever model I already pay for; I only need plausible inputs, not labelled outputs. Write the three reward functions against my real schemas, with the required-key gating exactly as the post describes. Run 100 GRPO steps on a 350M or 1B model on a rented GPU for a couple of dollars. Serve it behind llama.cpp with a JSON grammar so the shape is guaranteed, and keep the frontier model as a fallback only for the rows that fail validation twice.
The cost math is what sold me. The current setup sends every ticket to a paid API. If a local model handles even 70% of tickets cleanly and the fallback handles the rest, the bill drops by more than half and the p50 latency drops with it. That's the kind of number a client understands, and it's the sort of thing I end up building for the businesses I work with, where the model choice is never the interesting part and the plumbing around it always is.
One thing to run this week
Clone the IFStruct repo, point ifstruct-eval at whatever small model you already have running locally, and read the error breakdown at the bottom. Not the headline percentage. The breakdown. If "required field missing" dominates, your problem is trainable with a schema validator as the reward. If "type mismatch" dominates, you probably need constrained decoding first. Either way, you'll know in twenty minutes what you're actually paying the big model to avoid, and whether a 350M model with three afternoon-sized reward functions could take that job off its hands.
Originally published at abrarqasim.com. I write there about React, PHP, Rust, Go and the AI tooling around them.
Top comments (2)
The reward weights are the actual finding here, more than the 7 points. Schema validity at 2.0 against parseability at 1.0 matches what we see: a response that parses fine and quietly drops a required key is the failure that reaches production, while a fence where you asked for raw gets caught at the boundary.
I read your pushback on the augmentations and came out agreeing with you. Appending 'return this in a fenced block' to 40% of prompts is not free generalisation, it is you encoding known failure modes into the training set by hand. Did the 20% bare-list rewrite come before or after you saw which categories the base model failed? Because that ordering is the difference between a method and a debugging session with a GPU attached.
Quick correction first: I didn't run this one. The training is from the Hugging Face post by Monigatti, Burtenshaw and Paniego; I wrote about it and poked at the numbers. So I can't tell you the ordering from the inside.
What I can tell you is what the post shows. The base model's error breakdown is printed before the training section, and the augmentations are justified in one sentence as closing "two gaps" between Nemotron and IFStruct. The two gaps named are fenced-vs-raw and bare-list-with-count, which are two of the top ten error categories in that breakdown. The post doesn't say "we saw the breakdown, then designed the augmentations", but I'd be surprised if it happened the other way round. Your read matches mine: this is a debugging session with a GPU attached, written up as a method. That's not a criticism of the authors, it's how most applied RL actually goes, but it does mean the 7 points don't transfer to a benchmark whose failure modes you haven't looked at yet.
The detail I'd point you at, since you care about the weights more than the headline: compare the two error breakdowns. After training, "required field missing" goes from 7228 to 7331 and "wrong item count" from 738 to 890. Both went up. The schema reward at 2.0 didn't move the failure mode it was aimed at. What moved was the format stuff: unclosed fences and missing fences drop out of the top ten entirely. So the 7 points mostly came from the 1.0-weighted parseability reward, not the 2.0-weighted schema one. That's the opposite of what the weights suggest and the post doesn't discuss it.