Poolside has spent the days since releasing Laguna S 2.1 repairing it in public. Its base chat template shipped with reasoning switched off by default and a 32,768-token generation cap, both since changed. It replaced a template include with an inline version so that quantised conversions resolve correctly, then re-released its own Q4 and Q8 builds to fix tool calls and thinking. Those commits explain most of the split reviews.
Key facts
- The fix that matters: the base template's default changed from thinking disabled to enabled, and a 32,768-token generation cap was removed - after users had already been testing it.
- When: the model shipped July 21, 2026; template and quantisation repairs landed through July 24.
- Who: Poolside, whose commit history is public on Hugging Face.
- Primary source: the base-template commit and the updated GGUF repository.
We covered the launch itself on July 21: a 118-billion-parameter mixture-of-experts coding model activating roughly 8 billion parameters per token, with a router selecting ten of 256 experts plus a shared one, mostly 512-token sliding-window attention interleaved with periodic global-attention layers, and a stated context window of just over a million tokens. Strong claimed results, a permissive licence, and weights in every format practitioners use.
What has happened since is a useful lesson in what "the model" even means for an agentic system. Poolside's reported scores - roughly seven in ten terminal-agent tasks, close to four in five multilingual software-engineering tasks - came from its own harness, with its own sandbox, up to 500 steps per task, and averaged attempts. It disclosed all of that, and published its full trajectory archive, which is more auditable than almost any competing release. But the thing users downloaded was not the thing that scored. It was the same weights wrapped in a chat template that disabled the reasoning the whole design depends on, capped at 32,768 generated tokens.
That cap is the tell. Poolside's own numbers show maximum thinking lifting its terminal-agent score from 60.4% to 70.2%, and its hardest software-engineering result from 16.5% to 40.4% - with those stronger runs associated with roughly 129,000 and 249,000 completion tokens per trajectory. A model whose entire advantage is tenacity, shipped with a cap at a fraction of the token budget its results require and reasoning off by default, will look exactly like the negative reports: it plans forever and never finishes.
And those reports were substantive, not just mockery. On Poolside's own Hugging Face discussions, a user running the NVFP4 build on an RTX Pro 6000 reported loops, broken thinking blocks and endless generation in a coding harness, with the DFlash draft model's speculative decoding acceptance collapsing after a few tokens - and disabling DFlash tanking throughput. An INT4 user reported the model repeatedly creating the same folders after a long run. Poolside issued revised weights for several formats, acknowledged FP8 looping, and posted a release-candidate branch. On Reddit, one tester described a reproducible issue-plan-code-review loop where the model never got past planning even after template updates, and another watched it inspect the same files for twenty minutes before eventually finding a real bug. That is a specific failure mode - persistent but insufficiently controlled - not incapability.
The positive reports are equally specific and equally configuration-dependent. One user reported Laguna solving a constrained code-transformation task after more than 60,000 thinking tokens, where their local Qwen runs had failed - though the solution contained a questionable integer-packing shortcut. Others reported no loops at all once they used the revised template with a current llama.cpp build, or that the Q8 build is solid for coding. One FP8 user's problem vanished entirely after updating vLLM and clearing cached artefacts. When the same weights produce both outcomes depending on the serving stack, the serving stack is part of the product.
Poolside said as much before any of this, in its own limitations section: the model can overfit to its native harness, so a near-identical third-party tool schema may cause it to use a remembered interface rather than the supplied one; nested tool arguments can be malformed; and it can think for long stretches without progress. Those admissions predict the reported failures precisely. This is not a hostile community misreading a good model.
The strongest sceptical take - asked bluntly on r/LocalLLaMA - is how anyone should trust benchmark results when the released templates and quantised builds needed immediate repair. The commit history justifies the suspicion. It does not establish fabricated scores. The narrower explanation covers the evidence better: the benchmarked system and the initially downloadable one diverged, and the divergence happened in exactly the components - template, tool parser, quantisation, draft model - that decide whether an agent can complete a long task.
The licence, worth restating because it has been mischaracterised, is OpenMDW-1.1: commercial and non-commercial use, modification and redistribution are permitted, redistributors must retain the agreement and applicable notices, and there is a patent-litigation retaliation clause. It is permissive, not public domain.
The transferable lesson is uncomfortable for how the industry evaluates open-weight models. For a chat model, weights plus a rough template is close enough. For an agentic coding model, the chat template, tool parser, quantisation format, draft model and harness are load-bearing - and a benchmark advantage can vanish entirely in the gap between the lab's harness and yours.
Originally published on Ground Truth, where every claim is checked against the primary source.
Top comments (0)