The SQL specialist could produce executable queries, but it wrapped them in prose. The target was simple to describe: keep execution accuracy and remove the wrapper.
The first ref-free SimPO run collapsed execution accuracy from 0.860 to 0.080. Reference-anchored DPO recovered it to 0.900. A higher-pressure retry reached 0.920 and drove training loss to 0.007.
Clean-SQL rate remained 0.000 in every run.
That result changed the diagnosis. The 108 supervised targets were already bare SELECT statements. The unwanted wrapper came from the base model's prose prior, and the tested rank-4 adapter did not override it. More preference pressure improved execution but did not repair output format.
The decision became retry-data and, more specifically, reconsider the generation path. Constrained decoding or stronger supervised pressure fit the observed failure better than another composed DPO run.
The run also exposed a binary-provenance problem: release and debug builds composed adapters differently. The factory now reproduces the frozen baseline with the exact binary before trusting a comparison.
The measurements and caveats are in https://posttrainllm.com/devlog.
Top comments (0)