<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Edwin Jongsma</title>
    <description>The latest articles on DEV Community by Edwin Jongsma (@edwin_jongsma_1739c848559).</description>
    <link>https://dev.to/edwin_jongsma_1739c848559</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F1622509%2Ffe2d4601-a11a-496f-97c2-657888d3306f.jpg</url>
      <title>DEV Community: Edwin Jongsma</title>
      <link>https://dev.to/edwin_jongsma_1739c848559</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/edwin_jongsma_1739c848559"/>
    <language>en</language>
    <item>
      <title>Scale the Model, Scale the Harness: Unlocking a 2 Leap in Mathematical Reasoning</title>
      <dc:creator>Edwin Jongsma</dc:creator>
      <pubDate>Thu, 19 Nov 2026 01:39:16 +0000</pubDate>
      <link>https://dev.to/edwin_jongsma_1739c848559/scale-the-model-scale-the-harness-unlocking-a-2-leap-in-mathematical-reasoning-1i20</link>
      <guid>https://dev.to/edwin_jongsma_1739c848559/scale-the-model-scale-the-harness-unlocking-a-2-leap-in-mathematical-reasoning-1i20</guid>
      <description>&lt;p&gt;&lt;em&gt;Originally published in &lt;a href="https://generativeai.pub/scale-the-model-scale-the-harness-unlocking-a-2-leap-in-mathematical-reasoning-231314ab3ea7" rel="noopener noreferrer"&gt;Generative AI on Medium&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;I spent months building a system that wraps a language model in a checking loop so it can prove hard math theorems. Every step gets verified with real math tools, the system remembers what worked, and a second pass tries to break the proof before it counts.&lt;/p&gt;

&lt;p&gt;Then I tested it on 300 hard math problems, and the results surprised me: the setup around the model matters as much as the model itself.&lt;/p&gt;

&lt;h2&gt;
  
  
  What it does
&lt;/h2&gt;

&lt;p&gt;The harness is three things:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;A checking loop.&lt;/strong&gt; Every intermediate claim is verified in code (SymPy, Z3, 50-digit mpmath). The solver works on a persistent scratchpad; a separate auditor runs in a fresh kernel and tries to kill the proof.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Memory.&lt;/strong&gt; After each problem, a reflection pass distills what worked into heuristics the system reuses, with zero weight fine-tuning.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Governed search.&lt;/strong&gt; No proof drafting on turn 1, context collapses to a clean blackboard after rejections, and an overtime gate rescues near-misses.&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  The results
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Model&lt;/th&gt;
&lt;th&gt;One pass&lt;/th&gt;
&lt;th&gt;With the harness&lt;/th&gt;
&lt;th&gt;Multiplier&lt;/th&gt;
&lt;th&gt;Failures rescued&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Qwen-3.8-27B (local, one RTX 5090)&lt;/td&gt;
&lt;td&gt;12.67%&lt;/td&gt;
&lt;td&gt;26.67%&lt;/td&gt;
&lt;td&gt;2.11x&lt;/td&gt;
&lt;td&gt;16%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Meta Muse 1.3&lt;/td&gt;
&lt;td&gt;22.00%&lt;/td&gt;
&lt;td&gt;46.33%&lt;/td&gt;
&lt;td&gt;2.11x&lt;/td&gt;
&lt;td&gt;31%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Gemini 3.8 Flash&lt;/td&gt;
&lt;td&gt;33.67%&lt;/td&gt;
&lt;td&gt;55.33%&lt;/td&gt;
&lt;td&gt;1.64x&lt;/td&gt;
&lt;td&gt;33%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;3-model portfolio&lt;/td&gt;
&lt;td&gt;41.33%&lt;/td&gt;
&lt;td&gt;65.67%&lt;/td&gt;
&lt;td&gt;1.59x&lt;/td&gt;
&lt;td&gt;41%&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The honest headline is the last column: the loop fixed 16–33% of the problems each model got wrong on its own. Flash is the best model and improves least in multiples (1.64x, not 2x) but most in failures rescued.&lt;/p&gt;

&lt;p&gt;The detail I'm proudest of: the local 27B catches factor-of-2 and boundary-case errors both big models miss. Five problems only Qwen solved; it beat Muse on 14 problems and Flash on 10. Flash is also 7.8x faster than Muse at the median solve (12.2 vs 95.0 minutes).&lt;/p&gt;

&lt;h2&gt;
  
  
  The honest limitations
&lt;/h2&gt;

&lt;p&gt;Proofs are graded by our own calibrated LLM judge ensemble, running in an isolated context — no human grading, no formal verification. All the eval logs are public so anyone can recount: certified counts reproduce exactly from the judgment files (Qwen 80, Muse 139, Flash 166).&lt;/p&gt;

&lt;h2&gt;
  
  
  Read the full story
&lt;/h2&gt;

&lt;p&gt;The full article is published in Generative AI on Medium: &lt;a href="https://generativeai.pub/scale-the-model-scale-the-harness-unlocking-a-2-leap-in-mathematical-reasoning-231314ab3ea7" rel="noopener noreferrer"&gt;https://generativeai.pub/scale-the-model-scale-the-harness-unlocking-a-2-leap-in-mathematical-reasoning-231314ab3ea7&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The code and all test results are open: &lt;a href="https://github.com/dionyziuz/proof_harness" rel="noopener noreferrer"&gt;https://github.com/dionyziuz/proof_harness&lt;/a&gt; — eval logs: &lt;a href="https://github.com/dionyziuz/proof_harness/tree/main/data/eval_results" rel="noopener noreferrer"&gt;https://github.com/dionyziuz/proof_harness/tree/main/data/eval_results&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;If you try it or spot a flaw in the method, I'd like to hear it — the comments here or an issue on the repo.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>machinelearning</category>
      <category>llm</category>
      <category>opensource</category>
    </item>
  </channel>
</rss>
