DEV Community: Silas Liu The latest articles on DEV Community by Silas Liu (@silasyl). https://dev.to/silasyl https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F3903331%2F5a715d7a-6759-44fa-95e9-bcd05606f8a3.jpg DEV Community: Silas Liu https://dev.to/silasyl en Kodah: 51% on SWE-bench Lite with GPT-5-mini at $0.045/issue Silas Liu Wed, 29 Apr 2026 01:31:51 +0000 https://dev.to/silasyl/kodah-51-on-swe-bench-lite-with-gpt-5-mini-at-0045issue-5gip https://dev.to/silasyl/kodah-51-on-swe-bench-lite-with-gpt-5-mini-at-0045issue-5gip <p>I built Kodah around a simple bet: a small model with the right <br> approach can match frontier models on real engineering tasks, <br> at a fraction of the cost.</p> <p>The result: Using GPT-5-mini, Kodah reaches 81% of Claude Opus 4.6's <br> resolve rate, at 1/38th of the cost.</p> <h2> Results from the full 300-issue evaluation on SWE-bench Lite </h2> <ul> <li>153/300 resolved (51.0%), 280/300 generated a valid patch (93.3%)</li> <li>psf/requests: 6/6 (100%), django/django: 67/114 (58.8%)</li> <li>Flask: 0/3: codebases with heavy runtime coupling are the main weakness</li> <li>75% of issues cost under $0.05. 90% of issues cost under $0.10.</li> <li>Total cost for all 300 issues: $13.59</li> </ul> <p><a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fqoaffwb5yj1y91fm5hes.png" class="article-body-image-wrapper"><img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fqoaffwb5yj1y91fm5hes.png" alt="Cost vs. performance comparison: Kodah (51% resolve rate, $0.045/issue) vs Claude Opus 4.6 Thinking (62.7%, ~$1.70), GPT-5 (54.3%, ~$1.25), and Devin (13.86%, $2.25-$6.75) on SWE-bench Lite" width="700" height="600"></a></p> <p>Full breakdown with per-repo results and cost distribution: <br> <a href="https://www.silasdata.com/kodah" rel="noopener noreferrer">https://www.silasdata.com/kodah</a></p> <h2> How it works </h2> <p>Send a repo URL and an issue description, get a diff back.<br> Available as an API at <a href="https://kodah.io/" rel="noopener noreferrer">kodah.io</a>. First 10 fixes free.</p> ai python webdev productivity