DEV Community

Breach Protocol
Breach Protocol

Posted on • Originally published at groundtruth.day

Poolside's Laguna S 2.1 is a small open coding agent with big benchmark claims

Poolside released Laguna S 2.1, an open-weight coding model whose headline trick is doing frontier-style agentic work with unusually little active compute, so it can run locally on a single high-end machine. The model is real and genuinely accessible, but its most-shared claims, that it beats DeepSeek V4 Pro and has the best tool calling, come from Poolside's own benchmark table, and the one substantive hands-on test so far flags a real weakness: it invents facts when evidence runs out.

Key facts

  • Laguna S 2.1 is a text-only mixture-of-experts model, about 118 billion total parameters but only roughly 8 billion active per token, with a stated 1-million-token context. Model card
  • It is released under the permissive OpenMDW-1.1 license, with BF16, FP8, NVFP4, INT4, GGUF, and MLX variants. Release post
  • Poolside's OpenRouter endpoint prices it at 10 cents input and 20 cents output per million tokens, about 29 percent below DeepSeek V4 Flash's fresh-token rates. DeepSeek pricing
  • One early local test measured 109 tokens per second on a single RTX Pro 6000 and confirmed three fabrications under pressure. Hands-on eval

Some context on why the parameter count matters. In a mixture-of-experts model, the network is split into many "experts" and only a few fire for each token, so a model can hold a lot of knowledge while staying cheap to run. Laguna's design pushes this hard: 256 routed experts plus one shared expert, but only about 8 billion parameters active at a time. That is what lets a 118-billion-parameter model fit on hardware an enthusiast or small team can actually own, which is the real story here, because most models this capable need a server rack.

What Poolside can credibly claim is long-horizon coding with tool use. Its release describes interleaved thinking between tool calls and a persistence bet: test, backtrack, and keep working. Turning on its "thinking" mode substantially raised its own reported scores on agentic coding tasks, but roughly doubled to tripled the tokens it consumed, so the persistence is real and expensive. As Poolside frames it, the model is built for "software engineering and agentic coding," not general chat.

The benchmark claims need care. Poolside's comparison table puts Laguna ahead of the entry it labels DeepSeek-V4-Pro Max on five of six coding and agent measures, but behind on Toolathlon, a tool-use test. Poolside also says it takes the maximum of vendor, leaderboard, and third-party figures and used its own harness for one benchmark, warning the scores are less comparable. So the "beats V4 Pro" and "best tool calling" lines that traveled across r/LocalLLaMA are community framing, not settled fact, and Poolside's own table contradicts the tool-calling claim.

Why it matters: the most useful evidence is the one careful hands-on test, from a tester who ran 160 tasks on a single RTX Pro 6000 with a private, deterministic harness. Their verdict is nuanced and worth the whole story. Laguna's tool mechanics were excellent, with a 0.89 tool-argument pass rate, six-deep tool chains, and clean structured output, the fastest 100-billion-plus model they had measured on that card. But when they deliberately removed the data the model needed, it did not stop; it invented financial figures and status details, three fabrications they hand-confirmed. Their conclusion: the tool plumbing is superb, but the model can confidently keep going after the evidence has run out.

That aligns with Poolside's own disclosed limitations, which include reusing a remembered tool interface instead of matching a slightly different schema, malformed nested JSON arguments, and long stretches of thinking before progress. The honest caveat is that the hands-on numbers came from one private harness, on NVFP4 weights with an FP8 cache, and its author says the harness was tuned around Qwen models, so non-Qwen scores are lower bounds. The briefing-safe read: Laguna S 2.1 is a serious, unusually accessible open coding agent, a strong pick for supervised local coding with tool validation and human review, but a premature choice for autonomous agents that must reliably admit they do not know.


Originally published on Ground Truth, where every claim is checked against the primary source.

Top comments (0)