Okay, this is going to sound dumb, but the most expensive line item in one of my client pipelines last quarter was a model being asked to return JSON. Not to reason about anything. Just to take a support ticket, pull out six fields, and give them back as an object that json.loads wouldn't choke on. We were paying frontier-model prices for a task a regex would nearly do, because the cheap models kept returning a notes field nobody asked for, or wrapping the whole thing in a code fence when we said not to, or forgetting a required key one time in five.
So a Hugging Face post from last week got my attention. Leonie Monigatti, Ben Burtenshaw and Sergio Paniego take a 350M parameter model, LFM2.5-350M, run 100 steps of GRPO on about 500 samples, and lift its structured-output score from 22.6% to 29.7% on the IFStruct benchmark. On a free Colab GPU. In under an hour.
Those numbers aren't impressive on their own. 29.7% is still failing seven times out of ten. What's interesting is the shape of the recipe, because once you see it you realise the thing you're training isn't the model. It's the three reward functions. And those are the part you can write yourself.
What "structured output" is actually measuring
The IFStruct benchmark is worth understanding before the training bit makes sense. It's open source at Liquid4All/ifstruct, and it's narrowly about one thing: does the model return valid, parseable output in the format and shape you asked for? Not "is the answer right". Just "can this be wired into the next system without a human touching it".
The base model's failure breakdown on 2,000 test cases is where I'd start if I were you. 7,228 instances of a required field missing. 738 wrong item counts. 540 type mismatches. 317 unclosed code blocks. 190 cases of an extra notes field, 181 of an extra path, 175 of an extra constraints. The model isn't failing at understanding the schema. It's failing at discipline. It keeps adding helpful fields, keeps miscounting list items, keeps opening a fence and not closing it.
That's exactly the failure mode I was paying to avoid. And it's a much narrower problem than "make the model smarter", which is why a small model with a light nudge can move on it.
The recipe, minus the parts that don't matter
The post uses TRL's GRPO trainer. If you haven't touched GRPO, the short version: for each prompt, sample several completions (here, 8), score each one with reward functions, and push the model toward the completions that scored above the group average. No separate reward model, no preference pairs, no human labels. Just a scoring function that returns a number.
The LoRA config is nothing exotic, other than targeting LFM2.5's hybrid attention/convolution module names rather than the usual q_proj/v_proj pair:
lora_config = LoraConfig(
r=16,
lora_alpha=32,
task_type="CAUSAL_LM",
target_modules=["q_proj", "k_proj", "v_proj", "out_proj",
"in_proj", "w1", "w2", "w3"],
)
About 6M trainable parameters, 1.66% of the model. The training config is sized for a 16GB card:
training_args = GRPOConfig(
learning_rate=5e-5,
max_steps=100,
num_generations=8,
per_device_train_batch_size=4,
gradient_accumulation_steps=8,
max_completion_length=1024,
temperature=1.1,
beta=0.01,
reward_weights=[1.0, 0.5, 2.0],
)
Two of those numbers I'd flag. temperature=1.1 is hotter than you'd normally sample, and it's deliberate: GRPO needs the 8 completions in a group to actually differ, or there's no signal in "above average". And beta=0.01 is a small KL penalty toward the reference model, which is what stops 100 steps of "maximise the JSON score" from turning the model into something that only speaks JSON.
The rest of the pipeline is the TRL GRPO trainer doing what it does. I won't walk through it; the notebook is public and it runs.
The reward functions are the product
Here's the bit I keep coming back to. The three rewards, each on a 0 to 1 scale:
The first checks whether the output parses and is in the requested form. Fenced when asked for fenced, raw when asked for raw. Full credit for the right form, 0.2 for parseable but wrong form, zero for unparseable.
The second checks whether the top-level object has the expected number of fields, with the score decaying linearly as the count drifts.
The third validates the output against the row's JSON Schema, counts every violation, and gates partial credit on whether the required keys are present.
They're combined with weights of 1.0, 0.5 and 2.0. Schema validity is worth twice as much as parseability, which matches my experience of what actually breaks downstream.
Look at what's in there and what isn't. There's nothing about correctness. Nothing about whether the extracted invoice total is the right number. Nothing a human had to label. Every reward is a mechanical check you could write in an afternoon with json.loads and jsonschema.validate. Which means the "training data" for this is really just a pile of schemas and prompts, and the expensive part of fine-tuning, the labelled outputs, isn't needed at all.
If you've ever tried to build a preference dataset for something like this, that's the whole ballgame. I wrote about the moment a small fine-tuned model beat a frontier one on a narrow task and the labelling cost was the part that made it a hard sell to clients. Verifiable rewards remove that cost for any task where "did it comply" is a function you can write.
Where I'd push back
I don't want to oversell a 7-point lift. A few things bother me.
The gains are uneven and the post is upfront about the reason: the training data (NVIDIA's Nemotron structured-output set) doesn't look like the eval set, so the authors patch the gap by hand. 40% of prompts get a "return this in a fenced block" instruction appended, and a separate 20% are rewritten as bare-list tasks with a required item count. Those two augmentations map directly onto two of the base model's worst failure categories. That's smart, but it's also a reminder that the reward function isn't magic. You still have to know what the model gets wrong and put examples of it in front of the trainer.
The second thing: 29.7% is still a coin flip weighted against you. For a real pipeline I'd pair a model like this with strict decoding on the serving side, or with a retry loop that feeds validation errors back in. I covered why strict mode alone doesn't get you there with the big providers, and the same applies here in reverse. Constrained decoding guarantees the shape. It doesn't stop the model from filling the shape with nonsense. Training moves the model toward wanting to produce the right shape, which makes the constrained version's job easier and the failure rate on the unconstrained cases lower. You want both.
Third, and this is the honest one: the eval was run through llama.cpp on a MacBook, at BF16, with a specific --alias and context setting. The authors got 22.6% on the base model where the IFStruct paper reported 21.1%. That's close enough, but it tells you the number moves with the serving stack. Don't quote benchmark deltas across different servers.
What I'd actually do with this
For the support-ticket pipeline I opened with, here's the plan I've sketched and will try this month. Take the 40 or so ticket schemas we use in production. Generate a few hundred synthetic tickets per schema with whatever model I already pay for; I only need plausible inputs, not labelled outputs. Write the three reward functions against my real schemas, with the required-key gating exactly as the post describes. Run 100 GRPO steps on a 350M or 1B model on a rented GPU for a couple of dollars. Serve it behind llama.cpp with a JSON grammar so the shape is guaranteed, and keep the frontier model as a fallback only for the rows that fail validation twice.
The cost math is what sold me. The current setup sends every ticket to a paid API. If a local model handles even 70% of tickets cleanly and the fallback handles the rest, the bill drops by more than half and the p50 latency drops with it. That's the kind of number a client understands, and it's the sort of thing I end up building for the businesses I work with, where the model choice is never the interesting part and the plumbing around it always is.
One thing to run this week
Clone the IFStruct repo, point ifstruct-eval at whatever small model you already have running locally, and read the error breakdown at the bottom. Not the headline percentage. The breakdown. If "required field missing" dominates, your problem is trainable with a schema validator as the reward. If "type mismatch" dominates, you probably need constrained decoding first. Either way, you'll know in twenty minutes what you're actually paying the big model to avoid, and whether a 350M model with three afternoon-sized reward functions could take that job off its hands.
Originally published at abrarqasim.com. I write there about React, PHP, Rust, Go and the AI tooling around them.
Top comments (0)