<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Jay Pokale</title>
    <description>The latest articles on DEV Community by Jay Pokale (@jaypokale).</description>
    <link>https://dev.to/jaypokale</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4145200%2Ffaac088a-cfd1-4bfd-b1e7-f155cf2dd81e.jpg</url>
      <title>DEV Community: Jay Pokale</title>
      <link>https://dev.to/jaypokale</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/jaypokale"/>
    <language>en</language>
    <item>
      <title>Finding two ClickHouse time zone bugs with a differential test</title>
      <dc:creator>Jay Pokale</dc:creator>
      <pubDate>Mon, 16 Nov 2026 07:21:22 +0000</pubDate>
      <link>https://dev.to/jaypokale/finding-two-clickhouse-time-zone-bugs-with-a-differential-test-3k1n</link>
      <guid>https://dev.to/jaypokale/finding-two-clickhouse-time-zone-bugs-with-a-differential-test-3k1n</guid>
      <description>&lt;p&gt;I wanted a cheap way to find real bugs in a large database, so I stopped reading code and compared answers instead. Two of the comparisons found genuine bugs in ClickHouse's date functions, both in time zones with a UTC offset that is not a whole number of hours. The other three comparisons found nothing, which is also worth knowing.&lt;/p&gt;

&lt;p&gt;Both fixes are open pull requests, not merged. I say so up front because the title of a post like this tends to promise more than the state of the work.&lt;/p&gt;

&lt;h2&gt;
  
  
  The method: ask the same question two ways
&lt;/h2&gt;

&lt;p&gt;A differential test needs no expected values. You run the same logical query through two routes that must agree, and print any disagreement:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Against a reference.&lt;/strong&gt; ClickHouse's date functions against Python's &lt;code&gt;zoneinfo&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Against itself.&lt;/strong&gt; The same &lt;code&gt;JOIN&lt;/code&gt; with every join algorithm. The same filter with the primary key, skip indexes and &lt;code&gt;PREWHERE&lt;/code&gt; on and off. The same &lt;code&gt;GROUP BY&lt;/code&gt; with two-level, external and in-order aggregation.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Each probe is about 40-70 lines of Python that shells out to &lt;code&gt;clickhouse local&lt;/code&gt;. They are public gists, with the setup in the header: &lt;a href="https://gist.github.com/JayPokale/2c06ca77cf8b3cbc046eab637e460fb6" rel="noopener noreferrer"&gt;date functions&lt;/a&gt;, &lt;a href="https://gist.github.com/JayPokale/6f5ae4c3070b50a2f692182ee1b32c0d" rel="noopener noreferrer"&gt;JOIN algorithms&lt;/a&gt;, &lt;a href="https://gist.github.com/JayPokale/8433c09f6830ee38e95778cd5f4ee964" rel="noopener noreferrer"&gt;index analysis&lt;/a&gt;, &lt;a href="https://gist.github.com/JayPokale/e5287ff0949db5573fc4ff4bce22defc" rel="noopener noreferrer"&gt;aggregation settings&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Bug 1: &lt;code&gt;toStartOfHour&lt;/code&gt; returns &lt;code&gt;:30&lt;/code&gt; on a day when the clocks move by 30 minutes
&lt;/h2&gt;

&lt;p&gt;&lt;code&gt;Australia/Lord_Howe&lt;/code&gt; shifts by 30 minutes twice a year. The probe compares &lt;code&gt;toStartOfHour&lt;/code&gt; over thousands of timestamps with what Python says the wall clock hour is. Only two samples out of 4,000 disagreed, both on a transition day:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="nf"&gt;toStartOfHour&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;toDateTime&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'2024-04-07 03:10:00'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'Australia/Lord_Howe'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="c1"&gt;-- 2024-04-07 02:30:00   (expected 03:00:00)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Every other sample, including all the other days of the year in that zone, was right. The test only found this because it sampled around transitions on purpose. The cause is that &lt;code&gt;toStartOfHour&lt;/code&gt; ends in a function that rounds on a grid anchored at the local midnight. After a 30 minute shift, that grid no longer lines up with the clock. &lt;code&gt;toStartOfInterval(..., INTERVAL 2 HOUR)&lt;/code&gt; already compensated for the shift, so the 2-hour answer was right and the 1-hour answer wrong for the same input.&lt;/p&gt;

&lt;p&gt;The fix applies the same compensation to the 1-hour case, only when the day's shift is not a whole number of hours.&lt;/p&gt;

&lt;p&gt;Two things I learned from the CI run:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;I could not build ClickHouse here, so CI was the first time the test ran. It passed.&lt;/li&gt;
&lt;li&gt;The performance comparison, which is part of the CI, flagged &lt;code&gt;date_time_long&lt;/code&gt; at +51.7% on my first push. I had added a second lookup of the day to the hot path. Looking the day up once removed it. A correct fix can still be a regression, and the benchmark caught it before a reviewer had to.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Pull request: &lt;a href="https://github.com/ClickHouse/ClickHouse/pull/131620" rel="noopener noreferrer"&gt;ClickHouse #131620&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Bug 2: minute intervals before 1970 round to the wrong boundary
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;SET&lt;/span&gt; &lt;span class="n"&gt;enable_extended_results_for_datetime_functions&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="nf"&gt;toStartOfFifteenMinutes&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;toDateTime64&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'1910-06-15 12:07:00'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'Europe/Warsaw'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;          &lt;span class="c1"&gt;-- 12:00:00&lt;/span&gt;
&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="nf"&gt;toStartOfInterval&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;toDateTime64&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'1910-06-15 12:07:00'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'Europe/Warsaw'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kt"&gt;INTERVAL&lt;/span&gt; &lt;span class="mi"&gt;15&lt;/span&gt; &lt;span class="no"&gt;MINUTE&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;  &lt;span class="c1"&gt;-- 11:54:00&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;toStartOfInterval&lt;/code&gt; has an arithmetic shortcut that rounds the Unix timestamp. Before 1970 it was allowed whenever the offsets were whole minutes. The dedicated functions round from the local midnight there. The two agree only if the offset is a multiple of the interval, and Warsaw was at +01:24 until 1915. Checking the neighbours turned up &lt;code&gt;Pacific/Chatham&lt;/code&gt; (+12:45), where even &lt;code&gt;INTERVAL 10 MINUTE&lt;/code&gt; was wrong in 1960.&lt;/p&gt;

&lt;p&gt;The fix narrows the condition: whole hours that divide into the interval. Other zones take the slower generic path before the epoch, which is what the dedicated functions use.&lt;/p&gt;

&lt;p&gt;Pull request: &lt;a href="https://github.com/ClickHouse/ClickHouse/pull/131915" rel="noopener noreferrer"&gt;ClickHouse #131915&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  What found nothing
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;JOIN algorithms&lt;/strong&gt; (&lt;code&gt;hash&lt;/code&gt;, &lt;code&gt;parallel_hash&lt;/code&gt;, &lt;code&gt;grace_hash&lt;/code&gt;, &lt;code&gt;full_sorting_merge&lt;/code&gt;, &lt;code&gt;partial_merge&lt;/code&gt;, &lt;code&gt;direct&lt;/code&gt;, &lt;code&gt;auto&lt;/code&gt;) over NULL, NaN, tuple, array, map, &lt;code&gt;LowCardinality&lt;/code&gt; and decimal keys: no disagreement over 7 random seeds.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Index analysis&lt;/strong&gt; (primary key, three skip index types, &lt;code&gt;PREWHERE&lt;/code&gt;, the old analyzer, the query condition cache) against a full scan over 14 key types: about 6,000 comparisons, no disagreement.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Aggregation&lt;/strong&gt; settings against &lt;code&gt;GROUP BY&lt;/code&gt;, &lt;code&gt;DISTINCT&lt;/code&gt;, &lt;code&gt;ROLLUP&lt;/code&gt;, window functions on composite keys: about 3,200 comparisons, no disagreement.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;That is a useful result about the database. It is also a reminder that most of a mature system is correct, and the bugs live in the corners where two correct-looking pieces of logic meet, here "round to the hour" and "the offset changed by 30 minutes today".&lt;/p&gt;

&lt;h2&gt;
  
  
  The probes lied to me first
&lt;/h2&gt;

&lt;p&gt;Most of my debugging time went on the harness, not on ClickHouse:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;A first run reported thousands of mismatches. &lt;code&gt;DateTime&lt;/code&gt; is unsigned, so negative timestamps clamp to 1970, and my Python side disagreed with that. I limited the range.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;toStartOfMonth&lt;/code&gt; returns a &lt;code&gt;Date&lt;/code&gt;, which has no time zone, so comparing its Unix timestamp with Python's local midnight is meaningless. I compared dates instead.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;-0.0&lt;/code&gt; and &lt;code&gt;0.0&lt;/code&gt; group together but print differently, so the probe flagged a representative choice as a bug. I removed &lt;code&gt;-0.0&lt;/code&gt; from the value pool.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;clickhouse local&lt;/code&gt; keeps tables between runs, and running several seeds in parallel locked its directory. I gave each run its own path and a &lt;code&gt;DROP TABLE IF EXISTS&lt;/code&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If your first differential test reports a flood of bugs, it is almost always the test.&lt;/p&gt;

&lt;h2&gt;
  
  
  Takeaways
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;A reference implementation in another language is a good oracle for date and time code, because the two have different bugs.&lt;/li&gt;
&lt;li&gt;Sample around the edges on purpose. The Lord Howe bug showed up in 2 of 4,000 timestamps.&lt;/li&gt;
&lt;li&gt;When a fix touches a hot path, read the performance results, not only the tests.&lt;/li&gt;
&lt;li&gt;Say whether your fix is merged. Mine are not, yet.&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>testing</category>
      <category>database</category>
      <category>opensource</category>
      <category>python</category>
    </item>
    <item>
      <title>git séance: I asked the ghost of `request` why it died, and made it cite its sources</title>
      <dc:creator>Jay Pokale</dc:creator>
      <pubDate>Thu, 29 Oct 2026 03:49:19 +0000</pubDate>
      <link>https://dev.to/jaypokale/git-seance-i-asked-the-ghost-of-request-why-it-died-and-made-it-cite-its-sources-424g</link>
      <guid>https://dev.to/jaypokale/git-seance-i-asked-the-ghost-of-request-why-it-died-and-made-it-cite-its-sources-424g</guid>
      <description>&lt;p&gt;&lt;em&gt;This is a submission for the &lt;a href="https://dev.to/challenges/hacktoberfest-week4-2026-10-26"&gt;Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  What I Built
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://github.com/request/request" rel="noopener noreferrer"&gt;&lt;code&gt;request&lt;/code&gt;&lt;/a&gt; was the HTTP library of Node.js for a decade. In 2019 its author wrote&lt;br&gt;
&lt;a href="https://github.com/request/request/issues/3142" rel="noopener noreferrer"&gt;#3142&lt;/a&gt;, "Request's Past, Present and Future", and in 2020 it went into&lt;br&gt;
maintenance mode. The code is still on npm. The &lt;em&gt;reasons&lt;/em&gt; are scattered across 3,477 issues and pull requests, 12,441&lt;br&gt;
comments and 2,290 commits, and the people who knew them have moved on.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;git séance&lt;/strong&gt; lets you ask that history a question. A local open-weight model (Gemma 4 through Ollama) answers as the&lt;br&gt;
project's ghost, but it may only speak from the history: every claim cites an issue (&lt;code&gt;[#3142]&lt;/code&gt;) or a commit&lt;br&gt;
(&lt;code&gt;[b66e4cd]&lt;/code&gt;), and the code removes any citation that isn't in what was actually retrieved.&lt;/p&gt;

&lt;p&gt;It's for anyone who inherits or depends on an abandoned project and wants to know &lt;em&gt;why&lt;/em&gt; it is the way it is, before&lt;br&gt;
they "fix" something that was a deliberate decision ten years ago.&lt;/p&gt;
&lt;h2&gt;
  
  
  Demo
&lt;/h2&gt;


&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;      (
     ( )
      |
    .-'-.
    |   |
    '---'
You light a candle for request/request (2011-2026).
3477 threads, 12441 comments and 2290 commits stir in the dark. Ask, and they will answer.

you › How did request get automatic gzip decompression of responses?

The spirits whisper that the automatic decompression began with a dedicated effort to "Add support for gzip
content decoding" [#951]. This feature was implemented via a pull request, merging into the project history with
the commit [b66e4cd]. The goal was to allow the library to "Decompress 'x-gzip' content-encoding responses when
{gzip: true}" [#2446], enabling the module to handle responses that had a `Content-Encoding` header [#1680]. Later,
the project refined this process, introducing "More lenient gzip decompression" to ensure that "malformed but
otherwise acceptable server responses are still properly decompressed" [#2492].
   [#2492] thread by addaleax, 2016-12-21  https://github.com/request/request/pull/2492
   [#2446] thread by ghost, 2016-11-02  https://github.com/request/request/issues/2446
   [#1680] comment by simov, 2015-07-23  https://github.com/request/request/issues/1680#issuecomment-124089311
   [#951] thread by kevinoid, 2014-07-01  https://github.com/request/request/pull/951
   [b66e4cd] commit by Mikeal Rogers, 2014-07-02  https://github.com/request/request/commit/b66e4cd
   39s
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;p&gt;That's a real, unedited run on my laptop. The right answer is PR #951 by kevinoid, merged in July 2014, after three&lt;br&gt;
earlier attempts (#303, #458, #464) were closed. The answer isn't perfect: the "x-gzip" quote comes from a later issue&lt;br&gt;
(#2446), not the original feature. The citation is real, so the checker can't catch that; it only guarantees that&lt;br&gt;
every citation points at something that exists.&lt;/p&gt;
&lt;h2&gt;
  
  
  Code
&lt;/h2&gt;


&lt;div class="ltag-github-readme-tag"&gt;
  &lt;div class="readme-overview"&gt;
    &lt;h2&gt;
      &lt;img src="https://assets.dev.to/assets/github-logo-5a155e1f9a670af7944dd5e12375bc76ed542ea80224905ecaf878b9157cdefc.svg" alt="GitHub logo"&gt;
      &lt;a href="https://github.com/JayPokale" rel="noopener noreferrer"&gt;
        JayPokale
      &lt;/a&gt; / &lt;a href="https://github.com/JayPokale/git-seance" rel="noopener noreferrer"&gt;
        git-seance
      &lt;/a&gt;
    &lt;/h2&gt;
    &lt;h3&gt;
      Ask a dead open-source project why it did what it did. GraphRAG over its history with LanceDB, lance-graph and a local Gemma; every claim cites an issue or commit.
    &lt;/h3&gt;
  &lt;/div&gt;
  &lt;div class="ltag-github-body"&gt;
    
&lt;div id="readme" class="md"&gt;&lt;div class="markdown-heading"&gt;
&lt;h1 class="heading-element"&gt;git séance 🕯️&lt;/h1&gt;
&lt;/div&gt;
&lt;p&gt;Ask a dead open-source project why it did what it did, and get an answer that cites its own history.&lt;/p&gt;
&lt;p&gt;&lt;code&gt;git séance&lt;/code&gt; turns a repository's issues, pull requests, comments and commits into a graph, and answers questions
with a local open-weight model (Gemma 4 through Ollama) that may only speak from that history. Every claim cites an
issue (&lt;code&gt;[#3142]&lt;/code&gt;) or a commit (&lt;code&gt;[a1b2c3d]&lt;/code&gt;), and the code removes any citation that isn't in the retrieved evidence.&lt;/p&gt;
&lt;p&gt;The first project summoned is &lt;a href="https://github.com/request/request" rel="noopener noreferrer"&gt;&lt;code&gt;request&lt;/code&gt;&lt;/a&gt;, the Node.js HTTP library that went
into maintenance mode in 2020: 3,477 issues and pull requests, 12,441 comments and 2,290 commits.&lt;/p&gt;
&lt;div class="markdown-heading"&gt;
&lt;h2 class="heading-element"&gt;How it works&lt;/h2&gt;
&lt;/div&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Ingest&lt;/strong&gt; (&lt;code&gt;seance/ingest.py&lt;/code&gt;): issues, pull requests and their comments from the GitHub API, commits from a
blobless clone.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Build&lt;/strong&gt; (&lt;code&gt;seance/build.py&lt;/code&gt;): node tables &lt;code&gt;Thread&lt;/code&gt;, &lt;code&gt;Comment&lt;/code&gt;, &lt;code&gt;Commit&lt;/code&gt; and relationship tables &lt;code&gt;REPLIES&lt;/code&gt;
(comment → thread)…&lt;/li&gt;
&lt;/ol&gt;&lt;/div&gt;
  &lt;/div&gt;
  &lt;div class="gh-btn-container"&gt;&lt;a class="gh-btn" href="https://github.com/JayPokale/git-seance" rel="noopener noreferrer"&gt;View on GitHub&lt;/a&gt;&lt;/div&gt;
&lt;/div&gt;


&lt;h2&gt;
  
  
  How I Built It
&lt;/h2&gt;

&lt;h3&gt;
  
  
  History as a graph
&lt;/h3&gt;

&lt;p&gt;&lt;code&gt;ingest.py&lt;/code&gt; downloads issues, PRs and comments from the GitHub API and reads commits from a blobless clone.&lt;br&gt;
&lt;code&gt;build.py&lt;/code&gt; turns them into node tables (&lt;code&gt;Thread&lt;/code&gt;, &lt;code&gt;Comment&lt;/code&gt;, &lt;code&gt;Commit&lt;/code&gt;) and relationship tables, embeds every node with&lt;br&gt;
&lt;code&gt;nomic-embed-text&lt;/code&gt;, and stores it all in &lt;a href="https://github.com/lancedb/lancedb" rel="noopener noreferrer"&gt;LanceDB&lt;/a&gt;:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;relationship&lt;/th&gt;
&lt;th&gt;from → to&lt;/th&gt;
&lt;th&gt;where it comes from&lt;/th&gt;
&lt;th&gt;edges&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;REPLIES&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;comment → thread&lt;/td&gt;
&lt;td&gt;the comment's issue&lt;/td&gt;
&lt;td&gt;12,441&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;MENTIONS&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;commit → thread&lt;/td&gt;
&lt;td&gt;"Fixes #123" and friends in the message&lt;/td&gt;
&lt;td&gt;806&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;REFERS&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;thread → thread&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;#123&lt;/code&gt; and issue links in a thread or its comments&lt;/td&gt;
&lt;td&gt;1,675&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;PART_OF&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;commit → PR&lt;/td&gt;
&lt;td&gt;every commit of a merged PR's branch&lt;/td&gt;
&lt;td&gt;1,002&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The graph queries run with &lt;a href="https://github.com/lance-format/lance-graph" rel="noopener noreferrer"&gt;lance-graph&lt;/a&gt;, which executes Cypher over&lt;br&gt;
those Arrow tables. I've been sending fixes to lance-graph's planner, so I wanted to use it on something real:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight cypher"&gt;&lt;code&gt;&lt;span class="k"&gt;MATCH&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="ss"&gt;(&lt;/span&gt;&lt;span class="py"&gt;m:&lt;/span&gt;&lt;span class="k"&gt;Commit&lt;/span&gt;&lt;span class="ss"&gt;)&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="ss"&gt;[&lt;/span&gt;&lt;span class="nc"&gt;:PART_OF&lt;/span&gt;&lt;span class="ss"&gt;]&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="ss"&gt;(&lt;/span&gt;&lt;span class="py"&gt;t:&lt;/span&gt;&lt;span class="n"&gt;Thread&lt;/span&gt;&lt;span class="ss"&gt;)&lt;/span&gt; &lt;span class="k"&gt;WHERE&lt;/span&gt; &lt;span class="n"&gt;m.sha&lt;/span&gt; &lt;span class="ow"&gt;IN&lt;/span&gt; &lt;span class="ss"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'10246c8...'&lt;/span&gt;&lt;span class="ss"&gt;]&lt;/span&gt; &lt;span class="k"&gt;RETURN&lt;/span&gt; &lt;span class="n"&gt;t.number&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;number&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;(One honest wrinkle: lance-graph doesn't parse &lt;code&gt;[:MENTIONS|PART_OF]&lt;/code&gt; yet, so the code runs one query per&lt;br&gt;
relationship type.)&lt;/p&gt;

&lt;h3&gt;
  
  
  The first GraphRAG lost to plain search
&lt;/h3&gt;

&lt;p&gt;To measure anything, I wrote down 10 questions about &lt;code&gt;request&lt;/code&gt; and checked by hand which threads answer them, then&lt;br&gt;
scored plain vector search against graph retrieval with the &lt;strong&gt;same evidence budget&lt;/strong&gt; (16 items).&lt;/p&gt;

&lt;p&gt;My first graph version ranked whole threads and kept the top four. It did &lt;em&gt;worse&lt;/em&gt; than plain vector search: 5/10&lt;br&gt;
against 8/10. Two reasons. One-line comments like "same here" score higher than the thread that actually answers the&lt;br&gt;
question (0.83 against 0.71). And four threads were too few: vector search spreads its 16 items over a dozen threads.&lt;/p&gt;

&lt;p&gt;So the graph now &lt;strong&gt;adds to&lt;/strong&gt; vector search instead of replacing it. Ten slots go to the nearest threads, comments and&lt;br&gt;
commits; six go to what the graph says matters, ranking threads by how many of their parts matched, and adding their&lt;br&gt;
opening post and the commit that merged them.&lt;/p&gt;

&lt;h3&gt;
  
  
  The edge that mattered: commits belong to PRs
&lt;/h3&gt;

&lt;p&gt;The gzip question is the one plain search gets wrong, and it shows why the graph helps. The commit that added gzip,&lt;br&gt;
&lt;code&gt;10246c8 [PATCH v2] Add support for gzip content decoding&lt;/code&gt;, never mentions its PR. Only the merge commit does&lt;br&gt;
("Merge pull request #951"). The &lt;code&gt;PART_OF&lt;/code&gt; edge (from &lt;code&gt;git rev-list merge^1..merge^2&lt;/code&gt; for every merge commit) links&lt;br&gt;
the commit to PR #951, so the commit that vector search finds leads to the PR and its discussion.&lt;/p&gt;

&lt;h3&gt;
  
  
  Citations are checked in code
&lt;/h3&gt;

&lt;p&gt;Gemma has to end every claim with a citation. A regex then checks each one against the retrieved evidence and drops&lt;br&gt;
any that isn't there. That check had its own bug: Gemma writes citations as &lt;code&gt;[\#3142]&lt;/code&gt;, &lt;code&gt;[2780]&lt;/code&gt; and &lt;code&gt;[#12, #34]&lt;/code&gt;,&lt;br&gt;
and my first regex only accepted &lt;code&gt;[#3142]&lt;/code&gt;. It wasn't removing anything, but it wasn't counting anything either, so&lt;br&gt;
the first scores said Gemma cited the right thread in 4 of 10 answers. After fixing the parser and re-checking the&lt;br&gt;
same saved answers:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;10 questions, 16 evidence items&lt;/th&gt;
&lt;th&gt;answering thread retrieved&lt;/th&gt;
&lt;th&gt;answer cites it&lt;/th&gt;
&lt;th&gt;invented citations removed&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;vector search only&lt;/td&gt;
&lt;td&gt;8/10&lt;/td&gt;
&lt;td&gt;7/10&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;graph (lance-graph) + vector&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;9/10&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;9/10&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Gemma 4 E4B, about a minute per answer on a 6 GB GTX 1660 Ti.&lt;/p&gt;

&lt;p&gt;The caveats matter here:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;I chose the 10 questions and their answers myself, and I corrected one label (gzip) &lt;em&gt;after&lt;/em&gt; seeing what retrieval
found: I had first marked #303, which turned out to be a closed attempt, not the PR that landed.&lt;/li&gt;
&lt;li&gt;10 questions is small; treat these as directional, not a benchmark.&lt;/li&gt;
&lt;li&gt;Neither mode finds the thread where Node 0.8 support was dropped (#1427). It's a three-line issue, and dozens of
"doesn't work on node X" comments outscore it.&lt;/li&gt;
&lt;li&gt;"Invented citations removed: 0" means Gemma never cited something that wasn't in its evidence in these runs. It
doesn't mean every sentence is right, as the demo above shows.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Why Does Open Innovation Matter?
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;The project is open, so its whole memory is open.&lt;/strong&gt; Every issue, comment and commit of &lt;code&gt;request&lt;/code&gt; is public. That's
what makes a séance possible at all: a closed project's reasons die with its team.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;A local model makes re-asking free.&lt;/strong&gt; Every number above comes from running the same questions again and again
while changing the retrieval. On a paid API I'd have stopped after the first run looked fine, and I'd never have
seen that my first GraphRAG was worse than plain search.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Nothing to shut down.&lt;/strong&gt; The ghost runs on Gemma, LanceDB and lance-graph on one laptop. Unlike &lt;code&gt;request&lt;/code&gt; itself,
it can't be deprecated out from under you.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Where closed would win:&lt;/strong&gt; a frontier model would write smoother answers and would probably quote the right thread
more often. But the citation check matters more than fluency here: every answer can be checked against the history.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Prize Categories
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Best Use of Gemma&lt;/strong&gt;: Gemma 4 E4B (via Ollama) answers every question from retrieved evidence, with citations that
are checked in code, and is evaluated against plain vector search on hand-checked questions.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;em&gt;Built with heavy help from Claude Code (an AI coding agent): it wrote most of the code, chose and checked the&lt;br&gt;
evaluation questions, ran the evaluations, and drafted this post.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>devchallenge</category>
      <category>hf26challenge</category>
      <category>gemma</category>
      <category>opensource</category>
    </item>
    <item>
      <title>caveman vs ponytail vs Chisle: why I'd pick Chisle for Claude Code</title>
      <dc:creator>Jay Pokale</dc:creator>
      <pubDate>Fri, 23 Oct 2026 04:23:36 +0000</pubDate>
      <link>https://dev.to/jaypokale/caveman-vs-ponytail-vs-chisle-why-id-pick-chisle-for-claude-code-46lf</link>
      <guid>https://dev.to/jaypokale/caveman-vs-ponytail-vs-chisle-why-id-pick-chisle-for-claude-code-46lf</guid>
      <description>&lt;p&gt;If you've looked at cutting Claude Code's token use, you've probably found &lt;a href="https://github.com/JuliusBrussee/caveman" rel="noopener noreferrer"&gt;caveman&lt;/a&gt; and &lt;a href="https://github.com/DietrichGebert/ponytail" rel="noopener noreferrer"&gt;ponytail&lt;/a&gt;. Both are good. caveman makes Claude talk less. ponytail makes it build less.&lt;/p&gt;

&lt;p&gt;I started &lt;a href="https://github.com/JayPokale/Chisle" rel="noopener noreferrer"&gt;Chisle&lt;/a&gt; as a mashup of the two, and both are credited in the README. Here's why I think it's now the better pick.&lt;/p&gt;

&lt;h2&gt;
  
  
  1. It gives the shortest answers
&lt;/h2&gt;

&lt;p&gt;Same model, same prompts, only the ruleset changes. Answer length as a percentage of the model with no ruleset (lower is leaner):&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;caveman&lt;/th&gt;
&lt;th&gt;ponytail&lt;/th&gt;
&lt;th&gt;Chisle&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Claude Code, 26 prompts&lt;/td&gt;
&lt;td&gt;65%&lt;/td&gt;
&lt;td&gt;58%&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;56%&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Pi, 6 tasks&lt;/td&gt;
&lt;td&gt;56%&lt;/td&gt;
&lt;td&gt;49%&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;37%&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Here's one prompt from that set: "Implement an order state machine covering created, paid, shipped, delivered, cancelled and refunded."&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;no ruleset: 1,342 tokens, 125 lines, two classes, a custom exception and docstrings&lt;/li&gt;
&lt;li&gt;caveman: 1,140&lt;/li&gt;
&lt;li&gt;ponytail: 842&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Chisle: 450&lt;/strong&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Chisle's answer is one dict of legal transitions and one method that checks it:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;OrderStateMachine&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;TRANSITIONS&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;created&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;paid&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;cancelled&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;
        &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;paid&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;shipped&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;refunded&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;
        &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;shipped&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;delivered&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;cancelled&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;
        &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;delivered&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;refunded&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;
        &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;cancelled&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nf"&gt;set&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt;
        &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;refunded&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nf"&gt;set&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;transition&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;new_state&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;new_state&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;TRANSITIONS&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;state&lt;/span&gt;&lt;span class="p"&gt;]:&lt;/span&gt;
            &lt;span class="k"&gt;raise&lt;/span&gt; &lt;span class="nc"&gt;ValueError&lt;/span&gt;&lt;span class="p"&gt;(...)&lt;/span&gt;
        &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;state&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;new_state&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;p&gt;Same behaviour. A third of the tokens.&lt;/p&gt;
&lt;h2&gt;
  
  
  2. It doesn't make Claude dumber
&lt;/h2&gt;

&lt;p&gt;The fair worry with any of these tools is that tokens are where the model thinks, so fewer tokens means worse answers. I tested that without an LLM judge:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;312 maths and coding problems, graded by exact match and hidden unit tests: no ruleset 155/156, Chisle 156/156&lt;/li&gt;
&lt;li&gt;120 agent runs on real repos, graded by hidden tests: 49/60 vs 48/60&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Statistically there's no difference (p = 1.000 on both). Shorter answers, same results.&lt;/p&gt;
&lt;h2&gt;
  
  
  3. It goes after the part of the bill the others don't touch
&lt;/h2&gt;

&lt;p&gt;This is the big one. When I measured 171 of my own Claude Code sessions, Claude's replies weren't where the tokens were. &lt;strong&gt;Tool output was 67.5% of the context&lt;/strong&gt;: build logs, test runs, giant grep results. And it doesn't get billed once. It rides along with every later request in the session.&lt;/p&gt;

&lt;p&gt;caveman and ponytail only shape what Claude writes. Chisle also has a hook that trims big tool output before Claude reads it:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;strips color codes and collapses repeated lines&lt;/li&gt;
&lt;li&gt;keeps the start and end of long output, plus any error lines from the middle&lt;/li&gt;
&lt;li&gt;replaces exact repeats with a one-line note&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;It never touches Read, Edit or Write, because edits need the exact bytes. On the output it does touch, it cuts about 46%. With &lt;code&gt;npx chisle mcp wrap&lt;/code&gt; the same thing works for MCP servers in Cursor, Windsurf, Cline, Kiro, Gemini CLI and others.&lt;/p&gt;

&lt;p&gt;There are also a few rules about reading: grep for the symbol first, read only the slice you need, and don't re-open a file that's already in context.&lt;/p&gt;
&lt;h2&gt;
  
  
  4. It's one install for every agent you use
&lt;/h2&gt;


&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;npx chisle
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;p&gt;It finds and sets up Claude Code, Cursor, Windsurf, Cline, Kiro, Codex, Gemini, Copilot, Copilot CLI, OpenCode, Antigravity, Hermes and Pi. Zero dependencies, MIT, and &lt;code&gt;npx chisle --uninstall&lt;/code&gt; puts everything back.&lt;/p&gt;
&lt;h2&gt;
  
  
  5. You can check every number
&lt;/h2&gt;

&lt;p&gt;Every raw transcript behind these tables is committed in &lt;a href="https://github.com/JayPokale/Chisle/tree/main/benchmarks/results" rel="noopener noreferrer"&gt;benchmarks/results&lt;/a&gt;, including the runs where Chisle loses. The main one: on Pi, ponytail's total bill came in slightly lower (59% vs 61%), because the bill also counts hidden reasoning. I'd rather show you that than have you find it.&lt;/p&gt;


&lt;div class="ltag-github-readme-tag"&gt;
  &lt;div class="readme-overview"&gt;
    &lt;h2&gt;
      &lt;img src="https://assets.dev.to/assets/github-logo-5a155e1f9a670af7944dd5e12375bc76ed542ea80224905ecaf878b9157cdefc.svg" alt="GitHub logo"&gt;
      &lt;a href="https://github.com/JayPokale" rel="noopener noreferrer"&gt;
        JayPokale
      &lt;/a&gt; / &lt;a href="https://github.com/JayPokale/Chisle" rel="noopener noreferrer"&gt;
        Chisle
      &lt;/a&gt;
    &lt;/h2&gt;
    &lt;h3&gt;
      Cut your AI coding agent's token bill on three axes: terse prose, YAGNI-first code, and tool-output compression. Claude Code, Pi, Cursor, Codex, Gemini + 4 more. Zero deps, published benchmarks including the runs it loses.
    &lt;/h3&gt;
  &lt;/div&gt;
  &lt;div class="ltag-github-body"&gt;
    
&lt;div id="readme" class="md"&gt;&lt;p&gt;
  &lt;a rel="noopener noreferrer" href="https://github.com/JayPokale/Chisle/assets/logo.png"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fraw.githubusercontent.com%2FJayPokale%2FChisle%2FHEAD%2Fassets%2Flogo.png" width="120" alt="Chisle"&gt;&lt;/a&gt;
&lt;/p&gt;
&lt;div class="markdown-heading"&gt;
&lt;h1 class="heading-element"&gt;Chisle&lt;/h1&gt;
&lt;/div&gt;

&lt;div class="markdown-heading"&gt;
&lt;h3 class="heading-element"&gt;Fewer tokens than caveman and ponytail. No accuracy lost.&lt;/h3&gt;
&lt;/div&gt;

&lt;p&gt;
  &lt;em&gt;Your AI talks less, builds less, reads less, and says more. Like a senior dev who bills by the syllable.&lt;/em&gt;
&lt;/p&gt;

&lt;p&gt;
  &lt;em&gt;The only tool in this class that publishes the runs where it lost. &lt;a href="https://jaypokale.me/writing/chisle-benchmarks-it-loses" rel="nofollow noopener noreferrer"&gt;Here's why.&lt;/a&gt;&lt;/em&gt;
&lt;/p&gt;

&lt;p&gt;
  &lt;a href="https://www.npmjs.com/package/chisle" rel="nofollow noopener noreferrer"&gt;&lt;img src="https://camo.githubusercontent.com/945719fdc04e4dd512464a1c4b8c272d8f922b663714fbc2df04726d89302f13/68747470733a2f2f696d672e736869656c64732e696f2f6e706d2f762f636869736c653f7374796c653d666c61742d73717561726526636f6c6f723d643738613363" alt="npm version"&gt;&lt;/a&gt;
  &lt;a rel="noopener noreferrer nofollow" href="https://camo.githubusercontent.com/66c14dc95961fe439b012284ee1eca2820dc7caffbdc3759fa55b24b9ac1fa75/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f776f726b73253230776974682d31332532306167656e74732d6437386133633f7374796c653d666c61742d737175617265"&gt;&lt;img src="https://camo.githubusercontent.com/66c14dc95961fe439b012284ee1eca2820dc7caffbdc3759fa55b24b9ac1fa75/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f776f726b73253230776974682d31332532306167656e74732d6437386133633f7374796c653d666c61742d737175617265" alt="Works with 13 agents"&gt;&lt;/a&gt;
  &lt;a href="https://github.com/JayPokale/Chisle/actions/workflows/test.yml" rel="noopener noreferrer"&gt;&lt;img src="https://camo.githubusercontent.com/cb31b8cb9b49c154085465ab33384d14cbcab51f40305f1103042cf71faa9b43/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f616374696f6e732f776f726b666c6f772f7374617475732f4a6179506f6b616c652f436869736c652f746573742e796d6c3f7374796c653d666c61742d737175617265266c6162656c3d4349" alt="CI"&gt;&lt;/a&gt;
  &lt;a rel="noopener noreferrer nofollow" href="https://camo.githubusercontent.com/dff1f6228a3ea413e99e8089b1bddc580ab3174dc8493ab36170a5b64df9311a/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f646570732d302d3264613434653f7374796c653d666c61742d737175617265"&gt;&lt;img src="https://camo.githubusercontent.com/dff1f6228a3ea413e99e8089b1bddc580ab3174dc8493ab36170a5b64df9311a/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f646570732d302d3264613434653f7374796c653d666c61742d737175617265" alt="Zero deps"&gt;&lt;/a&gt;
  &lt;a rel="noopener noreferrer nofollow" href="https://camo.githubusercontent.com/72a0e4d42cb8f86d5dab6fda16879e90e6ce7ef278183d655fca9a04aef575c5/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d4d49542d6437386133633f7374796c653d666c61742d737175617265"&gt;&lt;img src="https://camo.githubusercontent.com/72a0e4d42cb8f86d5dab6fda16879e90e6ce7ef278183d655fca9a04aef575c5/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d4d49542d6437386133633f7374796c653d666c61742d737175617265" alt="MIT"&gt;&lt;/a&gt;
  &lt;a href="https://github.com/JayPokale/Chisle/stargazers" rel="noopener noreferrer"&gt;&lt;img src="https://camo.githubusercontent.com/9171d8224b2072eaac59ca83f1a903b66385b07a6a55dc565872cbba7460d782/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f73746172732f4a6179506f6b616c652f436869736c653f7374796c653d736f6369616c" alt="Star Chisle on GitHub"&gt;&lt;/a&gt;
&lt;/p&gt;

&lt;p&gt;
  &lt;strong&gt;Built for Claude Code: answers about half as long, shorter than caveman and ponytail · 13 agents · zero dependencies · one command&lt;/strong&gt;
&lt;/p&gt;

&lt;p&gt;Chisle is a Claude Code plugin that makes Claude cheaper to run without making it dumber. It cuts what Claude writes: no filler, no hedging, no speculative abstractions, just the smallest code that works. It also cuts what Claude reads: a &lt;code&gt;PostToolUse&lt;/code&gt; hook trims oversized tool output by ~46% before it re-enters the context window, where it would be re-billed on every later request. In agent-loop tests Claude with Chisle passes the same tasks as Claude without it. One &lt;code&gt;npx chisle&lt;/code&gt; installs…&lt;/p&gt;&lt;/div&gt;


&lt;/div&gt;
&lt;br&gt;
  &lt;div class="gh-btn-container"&gt;&lt;a class="gh-btn" href="https://github.com/JayPokale/Chisle" rel="noopener noreferrer"&gt;View on GitHub&lt;/a&gt;&lt;/div&gt;
&lt;br&gt;
&lt;/div&gt;
&lt;br&gt;


&lt;p&gt;If you try it next to caveman or ponytail on your own work, I'd like to hear how it compares.&lt;/p&gt;

</description>
      <category>claudecode</category>
      <category>ai</category>
      <category>productivity</category>
      <category>opensource</category>
    </item>
    <item>
      <title>sloth-standup: a local Gemma reads my 43 open PRs overnight, and found the review I'd missed for a week</title>
      <dc:creator>Jay Pokale</dc:creator>
      <pubDate>Thu, 22 Oct 2026 07:22:01 +0000</pubDate>
      <link>https://dev.to/jaypokale/sloth-standup-a-local-gemma-reads-my-43-open-prs-overnight-and-found-the-review-id-missed-for-a-3jj8</link>
      <guid>https://dev.to/jaypokale/sloth-standup-a-local-gemma-reads-my-43-open-prs-overnight-and-found-the-review-id-missed-for-a-3jj8</guid>
      <description>&lt;p&gt;&lt;em&gt;This is a submission for the &lt;a href="https://dev.to/challenges/hacktoberfest-week3-2026-10-19"&gt;Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  What I Built
&lt;/h2&gt;

&lt;p&gt;I have 43 open pull requests in 23 repositories, 14 of them in ClickHouse. Keeping up means opening each one,&lt;br&gt;
scrolling past CI reports and bot summaries, and working out whether anything actually needs me. Most days nothing&lt;br&gt;
does. Some days one thing does, and it's buried.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;sloth-standup&lt;/strong&gt; does that reading overnight, on my laptop, with Gemma 4 through Ollama, and leaves me a short&lt;br&gt;
digest: what merged, what needs me (and why, with a link to the comment), and what can wait.&lt;/p&gt;

&lt;p&gt;It's for the contributor side: not a team's incoming PRs, but your own PRs spread across other people's projects,&lt;br&gt;
where the maintainers are busy and the only one tracking them is you.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fzn21urdzfdiug3v6400e.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fzn21urdzfdiug3v6400e.png" alt="The morning digest" width="800" height="686"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;In its first two days it found two things I had missed:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;a review comment on a ClickHouse PR, unanswered since the week before. It was a real bug: with case-insensitive
matching, a named struct nested inside an unnamed one silently lost its values. I fixed it the same morning.&lt;/li&gt;
&lt;li&gt;a PR of mine in Interledger's rhyza that had become conflicting the moment its sibling PR was merged.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;
  
  
  Demo
&lt;/h2&gt;

&lt;p&gt;The screenshot above is a real digest from 2026-10-22 (my PRs, unedited). It runs as a systemd user timer at 07:00,&lt;br&gt;
or when the laptop wakes up if it was asleep, and pops a desktop notification with the PRs that need you.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;ollama pull gemma4:e2b
python3 &lt;span class="nt"&gt;-m&lt;/span&gt; standup &lt;span class="nt"&gt;--hours&lt;/span&gt; 24 &lt;span class="nt"&gt;--notify&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;h2&gt;
  
  
  Code
&lt;/h2&gt;


&lt;div class="ltag-github-readme-tag"&gt;
  &lt;div class="readme-overview"&gt;
    &lt;h2&gt;
      &lt;img src="https://assets.dev.to/assets/github-logo-5a155e1f9a670af7944dd5e12375bc76ed542ea80224905ecaf878b9157cdefc.svg" alt="GitHub logo"&gt;
      &lt;a href="https://github.com/JayPokale" rel="noopener noreferrer"&gt;
        JayPokale
      &lt;/a&gt; / &lt;a href="https://github.com/JayPokale/sloth-standup" rel="noopener noreferrer"&gt;
        sloth-standup
      &lt;/a&gt;
    &lt;/h2&gt;
    &lt;h3&gt;
      A local Gemma reads your open pull requests overnight and leaves a morning digest of the few that need you
    &lt;/h3&gt;
  &lt;/div&gt;
  &lt;div class="ltag-github-body"&gt;
    
&lt;div id="readme" class="md"&gt;&lt;div class="markdown-heading"&gt;
&lt;h1 class="heading-element"&gt;sloth-standup 🦥&lt;/h1&gt;
&lt;/div&gt;

&lt;p&gt;Reads every open pull request you have, overnight, with a local open-weight model, and leaves you a short morning
digest of the few that actually need you.&lt;/p&gt;
&lt;p&gt;If you contribute to several projects, a morning starts with opening dozens of PR tabs: CI reports, bot comments
review threads. Most say nothing that needs you. sloth-standup reads them so you don't have to, and is built to be
&lt;strong&gt;honest about what it can't tell&lt;/strong&gt;: a failing check nobody has explained is listed as exactly that, not guessed at.&lt;/p&gt;
&lt;div class="snippet-clipboard-content notranslate position-relative overflow-auto"&gt;&lt;pre class="notranslate"&gt;&lt;code&gt;# Morning standup for @JayPokale
## 🎉 Merged (3)
## 🦥 Needs you (1)
- interledger/rhyza#355: Look at the failing checks (Check if admin changelog is up to date).
## 🧵 Still unanswered (0)
## 👀 FYI (2)
## 🔴 Checks failing, nobody said why (1)
## 🔒 Deploy preview not authorized (1)
## 💤 Quiet (36)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;
&lt;div class="markdown-heading"&gt;
&lt;h2 class="heading-element"&gt;Run it&lt;/h2&gt;

&lt;/div&gt;
&lt;p&gt;Needs…&lt;/p&gt;&lt;/div&gt;
  &lt;/div&gt;
  &lt;div class="gh-btn-container"&gt;&lt;a class="gh-btn" href="https://github.com/JayPokale/sloth-standup" rel="noopener noreferrer"&gt;View on GitHub&lt;/a&gt;&lt;/div&gt;
&lt;/div&gt;


&lt;p&gt;Python standard library only, plus the &lt;code&gt;gh&lt;/code&gt; CLI and Ollama.&lt;/p&gt;

&lt;h2&gt;
  
  
  How I Built It
&lt;/h2&gt;

&lt;h3&gt;
  
  
  The model reads; the code decides
&lt;/h3&gt;

&lt;p&gt;My first version asked Gemma one question per PR: "does this need the author?" It scored 8 of 11 on hand-labelled&lt;br&gt;
PRs, and its mistakes had a pattern. When a PR had a failing check &lt;em&gt;and&lt;/em&gt; a comment about something&lt;br&gt;
else, the model forgot the failing check. And it ignored my own earlier comment explaining that a failure was a known&lt;br&gt;
flaky test.&lt;/p&gt;

&lt;p&gt;So I took away everything GitHub already knows. The code decides:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;a merge conflict needs you; so does "changes requested", until the reviewer speaks again;&lt;/li&gt;
&lt;li&gt;a review thread whose last word isn't yours needs you, &lt;strong&gt;however old it is&lt;/strong&gt; (that's how the week-old comment
surfaced: a "what's new since yesterday" digest would never have shown it);&lt;/li&gt;
&lt;li&gt;a comment can only explain a failing check if it was written &lt;em&gt;after the latest commit&lt;/em&gt;;&lt;/li&gt;
&lt;li&gt;a failing check with no comment to read is listed as "nobody said why", never guessed at;&lt;/li&gt;
&lt;li&gt;rollup checks (&lt;code&gt;Mergeable Check&lt;/code&gt;), fork deploy previews and PRs untouched for six months are not news.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;I'm not the first to land on this split: &lt;a href="https://dev.to/projekta2/why-my-chrome-extension-uses-a-hybrid-ai-risk-score-instead-of-pure-ai-sorting-4lfo"&gt;PR Focus&lt;/a&gt;&lt;br&gt;
describes the same choice of a hybrid score over pure AI sorting for PR triage.&lt;/p&gt;

&lt;p&gt;Gemma answers only two narrow questions, with a JSON schema and temperature 0: &lt;em&gt;does a person or a review bot ask&lt;br&gt;
the author for something they haven't answered yet?&lt;/em&gt; and &lt;em&gt;has anyone explained the failing checks?&lt;/em&gt; Every name, link&lt;br&gt;
and number in the digest comes from GitHub's data, not from the model.&lt;/p&gt;

&lt;p&gt;Two small things mattered more than I expected. CI comments are full of links that are kilobytes of base64, and&lt;br&gt;
stripping link targets halved the model time and fixed misreadings. And the network is not always awake at 7 a.m.,&lt;br&gt;
so every GitHub call retries, and a PR that can't be read is listed as "couldn't read", not silently dropped.&lt;/p&gt;

&lt;h3&gt;
  
  
  Measuring it honestly
&lt;/h3&gt;

&lt;p&gt;&lt;code&gt;eval/&lt;/code&gt; holds saved snapshots of real PR activity with hand labels, and &lt;code&gt;eval/run.py&lt;/code&gt; scores any model against them,&lt;br&gt;
including &lt;code&gt;none&lt;/code&gt;: the rules alone.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;41 hand-labelled PRs&lt;/th&gt;
&lt;th&gt;correct&lt;/th&gt;
&lt;th&gt;needs you: caught&lt;/th&gt;
&lt;th&gt;false alarms&lt;/th&gt;
&lt;th&gt;model time&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;rules only, no model&lt;/td&gt;
&lt;td&gt;34/41&lt;/td&gt;
&lt;td&gt;16/17&lt;/td&gt;
&lt;td&gt;6/24&lt;/td&gt;
&lt;td&gt;0 s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Gemma 4 E2B&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;40/41&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;16/17&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;0/24&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;357 s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Gemma 4 E4B&lt;/td&gt;
&lt;td&gt;38/41&lt;/td&gt;
&lt;td&gt;16/17&lt;/td&gt;
&lt;td&gt;2/24&lt;/td&gt;
&lt;td&gt;473 s&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;(18 of my PRs, 9 of an AI agent account and 14 of a maintainer; laptop GTX 1660 Ti.)&lt;/p&gt;

&lt;p&gt;The caveats, because they matter:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;I tuned the rules and the prompt on 27 of these 41 PRs, so those numbers flatter the tool. The first time I tried
it on someone else's PRs (an AI agent account that contributes to ClickHouse) it scored &lt;strong&gt;1 of 9&lt;/strong&gt;: it treated an
aggregate check as a real failure, and an explanation written for an older commit as current. Both became rules.&lt;/li&gt;
&lt;li&gt;Then I labelled a fresh set, a ClickHouse maintainer's 14 most recently updated PRs, before running anything on
it, and ran it once: the rules alone got 14/14 and Gemma 13/14. Most of those PRs are decided by the open-thread
rule, so that set tests the rules more than the model.&lt;/li&gt;
&lt;li&gt;What the model adds is reading explanations: every false alarm of the rules-only run is a failure someone had
already explained in a comment. Its one held-out miss is the opposite mistake: it read an unrelated comment as an
explanation.&lt;/li&gt;
&lt;li&gt;The 2B-effective Gemma 4 E2B beat the 4B E4B here, and took half the time, so it's the default.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Why Does Open Innovation Matter?
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;It reads code review, which isn't mine to send anywhere.&lt;/strong&gt; The comments on my PRs are other people's words on
other people's projects. Running the model locally means none of that leaves my laptop.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Re-running is free, so I could measure instead of guess.&lt;/strong&gt; Every rule above came from running the model on the
same snapshots again and again, and comparing against the rules-only baseline. On a paid API I'd have stopped at
"looks good on today's digest".&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;It has to be boring to run.&lt;/strong&gt; A morning job that costs nothing and needs no key just keeps running. I set it up
once.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Where closed would win:&lt;/strong&gt; a frontier model would likely not need half of my rules. But the rules make the
digest predictable, and with them a small open model is enough.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  The Lazy Payoff
&lt;/h2&gt;

&lt;p&gt;On the morning of 2026-10-22 it covered 43 open PRs and the 30 hours since the previous check: 3 merged, 3 new&lt;br&gt;
comments, red checks on 6 PRs. Its answer: 2 need me, 5 are FYI with a one-line reason each, 1 is a deploy preview&lt;br&gt;
waiting for a maintainer, 34 have nothing new. Two tabs instead of 43.&lt;/p&gt;

&lt;p&gt;The bigger payoff was the first two days: the week-old unanswered review and the PR that had turned conflicting.&lt;br&gt;
Checking tabs by hand had missed both.&lt;/p&gt;

&lt;p&gt;It also changes what "done" means for a red check: once someone writes down why it failed (on the PR, after the&lt;br&gt;
latest commit), the digest stops flagging it, so the explanation helps the maintainer and my next morning alike.&lt;/p&gt;

&lt;h2&gt;
  
  
  Prize Categories
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Best Use of Gemma&lt;/strong&gt;: Gemma 4 (E2B by default, E4B tested) runs locally through Ollama, with JSON-schema output,
and is evaluated against hand labels and a rules-only baseline.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;em&gt;Built with heavy help from Claude Code (an AI coding agent): it wrote most of the code, labelled the evaluation&lt;br&gt;
PRs by the rule written at the top of &lt;code&gt;eval/labels.json&lt;/code&gt;, ran the evaluations, and drafted this post.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>devchallenge</category>
      <category>hf26challenge</category>
      <category>gemma</category>
      <category>opensource</category>
    </item>
    <item>
      <title>Dare2Solve Studio: fresh math puzzles a tutor can trust, written by a model on their own laptop</title>
      <dc:creator>Jay Pokale</dc:creator>
      <pubDate>Sun, 18 Oct 2026 05:24:47 +0000</pubDate>
      <link>https://dev.to/jaypokale/dare2solve-studio-fresh-math-puzzles-a-tutor-can-trust-written-by-a-model-on-their-own-laptop-2eka</link>
      <guid>https://dev.to/jaypokale/dare2solve-studio-fresh-math-puzzles-a-tutor-can-trust-written-by-a-model-on-their-own-laptop-2eka</guid>
      <description>&lt;p&gt;&lt;em&gt;This is a submission for the &lt;a href="https://dev.to/challenges/hf26"&gt;Hacktoberfest Open-Source AI Challenge: Week 2&lt;/a&gt; — Side Hustle&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  What I Built
&lt;/h2&gt;

&lt;p&gt;Every tutor in India who runs a small coaching class has the same weekly chore: find ten new questions for Saturday's&lt;br&gt;
test. Textbook questions are already solved on YouTube. Question banks cost money and still repeat. Asking a chatbot&lt;br&gt;
gives you fresh questions, but you then have to solve each one yourself, because a wrong answer key in front of a&lt;br&gt;
class costs you more than the time you saved.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Dare2Solve Studio&lt;/strong&gt; writes fresh math puzzles and refuses to hand over any puzzle whose answer it couldn't confirm.&lt;br&gt;
A tutor picks a topic (probability, remainders, time and distance...), an audience (classes 8–10, JEE, olympiad,&lt;br&gt;
placement aptitude) and a difficulty, and gets:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;puzzles that passed two independent checks, stored in a searchable bank that rejects near-duplicates,&lt;/li&gt;
&lt;li&gt;a printable worksheet PDF with a name/score header and an answer key with worked solutions,&lt;/li&gt;
&lt;li&gt;a square "puzzle of the day" card to post in the class WhatsApp group.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Who pays: a tutor or a small coaching centre making weekly tests, at about the price of a pizza a month&lt;br&gt;
(₹299/month in my head). That only works because the model costs nothing to run (more on that below).&lt;/p&gt;
&lt;h2&gt;
  
  
  Demo
&lt;/h2&gt;

&lt;p&gt;  &lt;iframe src="https://www.youtube.com/embed/MVTBaImAQBc" width="710" height="399"&gt;
  &lt;/iframe&gt;
&lt;/p&gt;

&lt;h2&gt;
  
  
  Code
&lt;/h2&gt;


&lt;div class="ltag-github-readme-tag"&gt;
  &lt;div class="readme-overview"&gt;
    &lt;h2&gt;
      &lt;img src="https://assets.dev.to/assets/github-logo-5a155e1f9a670af7944dd5e12375bc76ed542ea80224905ecaf878b9157cdefc.svg" alt="GitHub logo"&gt;
      &lt;a href="https://github.com/JayPokale" rel="noopener noreferrer"&gt;
        JayPokale
      &lt;/a&gt; / &lt;a href="https://github.com/JayPokale/dare2solve-studio" rel="noopener noreferrer"&gt;
        dare2solve-studio
      &lt;/a&gt;
    &lt;/h2&gt;
    &lt;h3&gt;
      Fresh math puzzles a tutor can trust: a local Gemma model writes them, code and a blind re-solve check every answer, Tiger Cloud keeps a deduplicated bank
    &lt;/h3&gt;
  &lt;/div&gt;
  &lt;div class="ltag-github-body"&gt;
    
&lt;div id="readme" class="md"&gt;&lt;div class="markdown-heading"&gt;
&lt;h1 class="heading-element"&gt;Dare2Solve Studio&lt;/h1&gt;
&lt;/div&gt;
&lt;p&gt;Fresh math puzzles for tutors and coaching classes, with every answer checked by running code
It runs an open-weight model (Gemma 4 E4B through Ollama) on your own computer.&lt;/p&gt;
&lt;p&gt;Demo (2 minutes): &lt;a href="https://youtu.be/MVTBaImAQBc" rel="nofollow noopener noreferrer"&gt;https://youtu.be/MVTBaImAQBc&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;A tutor picks a topic, an audience (classes 8–10, JEE, olympiad, placement aptitude) and a difficulty. The studio
writes new puzzles, rejects every one whose answer it can't confirm, and keeps the rest in a searchable bank. From
the bank it prints a worksheet PDF with an answer key, or a square "puzzle of the day" card for WhatsApp or Instagram.&lt;/p&gt;
&lt;div class="markdown-heading"&gt;
&lt;h2 class="heading-element"&gt;How a puzzle is checked&lt;/h2&gt;
&lt;/div&gt;
&lt;p&gt;The model writes five things for each puzzle: the problem, its working notes, a Python &lt;code&gt;solve()&lt;/code&gt; function that
computes the answer independently (by brute force where possible), a worked solution, and the answer. The notes get
their own field because a small model otherwise thinks inside the code: dozens…&lt;/p&gt;&lt;/div&gt;
  &lt;/div&gt;
  &lt;div class="gh-btn-container"&gt;&lt;a class="gh-btn" href="https://github.com/JayPokale/dare2solve-studio" rel="noopener noreferrer"&gt;View on GitHub&lt;/a&gt;&lt;/div&gt;
&lt;/div&gt;


&lt;h2&gt;
  
  
  How I Built It
&lt;/h2&gt;

&lt;h3&gt;
  
  
  A model that's wrong a lot, wrapped in checks that aren't
&lt;/h3&gt;

&lt;p&gt;The model is &lt;strong&gt;Gemma 4 E4B&lt;/strong&gt;, running in Ollama on my laptop's 6 GB GTX 1660 Ti. A small model (about&lt;br&gt;
4 billion effective parameters) writes fluent, well-set puzzles, and gets the arithmetic wrong surprisingly often. So the design assumes every&lt;br&gt;
answer is wrong until proven otherwise. For every puzzle the model returns JSON (enforced with Ollama's JSON-schema&lt;br&gt;
&lt;code&gt;format&lt;/code&gt;) with the problem, the answer, a worked solution and a Python &lt;code&gt;solve()&lt;/code&gt; function, and then:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;The code check.&lt;/strong&gt; &lt;code&gt;solve()&lt;/code&gt; must compute the answer on its own, by brute force when possible, and must
agree with the stated answer. It runs in a sandbox: an AST whitelist of imports, no dunder attributes, no
&lt;code&gt;open&lt;/code&gt;/&lt;code&gt;exec&lt;/code&gt;/&lt;code&gt;eval&lt;/code&gt;, an empty environment, and &lt;code&gt;RLIMIT&lt;/code&gt; caps on CPU, memory, file size and file handles.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The blind re-solve.&lt;/strong&gt; The model then sees &lt;em&gt;only the problem text&lt;/em&gt; and must first say whether it's well
posed, then solve it again with fresh code. (Its notes go in a separate &lt;code&gt;reasoning&lt;/code&gt; field; see below for why.) This catches the failure the code check can't: a puzzle that is
internally consistent with its own code but contradictory or missing a fact.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;A real solution.&lt;/strong&gt; At least three numbered steps, or it's rejected.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;When the model's own arithmetic slips but its program runs, and the blind re-solve agrees with the &lt;em&gt;program&lt;/em&gt;, two&lt;br&gt;
independent derivations agree, so the puzzle is kept: the answer is corrected and the solution rewritten for it.&lt;/p&gt;

&lt;h3&gt;
  
  
  What the checks caught
&lt;/h3&gt;

&lt;p&gt;I ran the pipeline on eight topics, one puzzle each, up to three attempts per puzzle, with Gemma 4 E4B:&lt;br&gt;
&lt;strong&gt;8 of 8 topics produced a puzzle that passed every check, and 5 of 13 attempts were discarded on the way&lt;/strong&gt;&lt;br&gt;
(about 30 minutes on the laptop). The discards are the point. In this run they were:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;two puzzles where the blind re-solve reached a different answer than the original,&lt;/li&gt;
&lt;li&gt;one the blind solver called ill-posed,&lt;/li&gt;
&lt;li&gt;one whose code disagreed with its own stated answer, and&lt;/li&gt;
&lt;li&gt;one whose worked solution had only one step.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Earlier runs also caught solver code that didn't compile and a ₹ sign &lt;em&gt;inside&lt;/em&gt; the Python code (the prompt asks&lt;br&gt;
for Indian context with money in ₹; it now asks for ASCII-only code).&lt;/p&gt;

&lt;p&gt;Then I checked the checker: I solved all eight puzzles by hand (and brute-forced them). &lt;strong&gt;Seven were correct. One&lt;br&gt;
had no solution at all&lt;/strong&gt;: both the original program and the blind one returned &lt;code&gt;None&lt;/code&gt;, and &lt;code&gt;None == None&lt;/code&gt;, so the&lt;br&gt;
pipeline counted it as two derivations agreeing. That's fixed: a &lt;code&gt;solve()&lt;/code&gt; that returns anything other than a number&lt;br&gt;
now fails. One of the seven also gives its own answer away (it asks for a digit sum it already stated), which no&lt;br&gt;
check catches yet; a tutor would skip that one.&lt;/p&gt;

&lt;p&gt;My own bugs were caught too. An output limit of 1,500 tokens silently truncated the JSON in a whole evaluation&lt;br&gt;
round; I threw that round's data away instead of reporting it. And in the round before this one, only 2 of 8 topics&lt;br&gt;
verified. Reading the raw replies showed why: the model was &lt;em&gt;thinking inside &lt;code&gt;solve_code&lt;/code&gt;&lt;/em&gt;, writing forty lines of&lt;br&gt;
comments ("Wait, let's re-read the problem…") and stopping before the function had a body. The checks rejected it&lt;br&gt;
as a syntax error, correctly, but it wasn't a wrong answer, just nowhere to think. Giving the JSON a separate&lt;br&gt;
&lt;code&gt;reasoning&lt;/code&gt; field ahead of the code, and asking for code "not notes", fixed it: the four problems that had failed&lt;br&gt;
this way all produced running code on the next try, and the numbers above are from the rerun with that fix.&lt;/p&gt;

&lt;h3&gt;
  
  
  A bank that doesn't repeat itself (Tiger Data)
&lt;/h3&gt;

&lt;p&gt;Verified puzzles go into a &lt;strong&gt;Tiger Cloud&lt;/strong&gt; hypertable with a 384-dimensional embedding of the problem text&lt;br&gt;
(&lt;code&gt;bge-small-en-v1.5&lt;/code&gt; via fastembed, also local) and an HNSW index. Before a puzzle is stored, the bank looks up its&lt;br&gt;
nearest neighbour; above 0.92 cosine similarity it's rejected as the same question in different words. The same&lt;br&gt;
index powers "search by meaning", so a tutor can type "probability with coloured balls" and find matching puzzles&lt;br&gt;
regardless of wording. Time-partitioning by creation date keeps "what did I make this month" cheap as the bank grows.&lt;/p&gt;

&lt;h3&gt;
  
  
  Being kind to a laptop
&lt;/h3&gt;

&lt;p&gt;My laptop froze twice while I was building this, under GPU heat and memory pressure. So the evaluation waits&lt;br&gt;
whenever the GPU is above 78 °C, pauses between puzzles, and the web app runs one generation at a time on one&lt;br&gt;
worker thread. Slow, but a tutor makes a worksheet once a week.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why Does Open Innovation Matter?
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;The business only works at zero marginal cost.&lt;/strong&gt; Each puzzle takes several model calls (write and re-solve,
sometimes a code retry or a rewrite, up to three attempts). On a paid API that's a cost per puzzle that a ₹299/month price can't carry for a
tutor making 50 puzzles a week. On an open-weight model on hardware the tutor already owns, it costs electricity.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Checks matter more than model size.&lt;/strong&gt; Because I could run the model as many times as I liked, I could afford
to distrust it: generate, verify with code, re-solve blind, retry. The open model plus cheap verification beats a
bigger model I'd have to trust.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;It works offline and stays private.&lt;/strong&gt; A tutor's question bank is their product. It never leaves their machine,
apart from the optional cloud bank they own.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Prize Categories
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Best Use of Gemma&lt;/strong&gt;: Gemma 4 E4B writes, solves and re-solves every puzzle, with schema-constrained JSON output.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Best Use of Tiger Data&lt;/strong&gt;: the puzzle bank is a Tiger Cloud hypertable with pgvector HNSW search and near-duplicate rejection.&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>devchallenge</category>
      <category>hf26challenge</category>
      <category>ai</category>
      <category>opensource</category>
    </item>
    <item>
      <title>My speech-flaw detector flagged 41 false alarms a minute. One line of math fixed it.</title>
      <dc:creator>Jay Pokale</dc:creator>
      <pubDate>Tue, 06 Oct 2026 20:04:50 +0000</pubDate>
      <link>https://dev.to/jaypokale/my-speech-flaw-detector-flagged-41-false-alarms-a-minute-one-line-of-math-fixed-it-1id1</link>
      <guid>https://dev.to/jaypokale/my-speech-flaw-detector-flagged-41-false-alarms-a-minute-one-line-of-math-fixed-it-1id1</guid>
      <description>&lt;p&gt;I built &lt;strong&gt;Podium&lt;/strong&gt;, a tool that compares your reading of a speech with a great delivery of the &lt;em&gt;same text&lt;/em&gt; (JFK, Reagan) and tells you exactly where you drift and why: "14.3 syllables/s here vs 7.7 in the reference (+85%)", pinned to a time range.&lt;/p&gt;

&lt;p&gt;The first version worked beautifully on my test set. Then I gave it a different speaker, and it flagged &lt;strong&gt;41 "flaws" per minute&lt;/strong&gt; on a perfectly clean recording.&lt;/p&gt;

&lt;p&gt;This post covers why that happened, the one-line fix, and the evaluation set-up that caught it.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fvvt909v4wwvxdpaqa4m6.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fvvt909v4wwvxdpaqa4m6.png" alt="Podium dashboard" width="800" height="1486"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 1: put both recordings on the same word grid
&lt;/h2&gt;

&lt;p&gt;Comparing two deliveries frame by frame is hopeless: they never line up. Comparing them &lt;em&gt;word by word&lt;/em&gt; is easy, as long as both are force-aligned to the same transcript. I used torchaudio's &lt;code&gt;MMS_FA&lt;/code&gt; (a wav2vec2 CTC aligner):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;bundle&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;torchaudio&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;pipelines&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;MMS_FA&lt;/span&gt;
&lt;span class="n"&gt;model&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;tokenizer&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;aligner&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;bundle&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get_model&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;with_star&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="bp"&gt;False&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt; &lt;span class="n"&gt;bundle&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get_tokenizer&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt; &lt;span class="n"&gt;bundle&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get_aligner&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;span class="n"&gt;emission&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;_&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;model&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;wav&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;spans&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;aligner&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;emission&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt; &lt;span class="nf"&gt;tokenizer&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;words&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;   &lt;span class="c1"&gt;# one span list per word
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;p&gt;Now word 17 of your reading is word 17 of JFK's. For every word I compute speaker-normalised features with Praat (via Parselmouth) and librosa:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;pitch in &lt;strong&gt;semitones relative to your own median&lt;/strong&gt; (a deep and a high voice with the same intonation give the same curve);&lt;/li&gt;
&lt;li&gt;loudness in &lt;strong&gt;dB relative to your own level&lt;/strong&gt;;&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;articulation rate&lt;/strong&gt;, syllables per second of actual speech, pauses excluded, over a 5-word window;&lt;/li&gt;
&lt;li&gt;the pause before the word, and &lt;strong&gt;voiced sound inside that pause&lt;/strong&gt; (more on that below);&lt;/li&gt;
&lt;li&gt;high-frequency energy above 2.5 kHz, which drops when consonants get swallowed.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;
  
  
  Step 2: build a dataset where the answers are exact
&lt;/h2&gt;

&lt;p&gt;There's no public dataset that pairs a good delivery with bad deliveries of the same words. So I made one: &lt;strong&gt;207 recordings&lt;/strong&gt;, starting from three public-domain speeches.&lt;/p&gt;

&lt;p&gt;The trick is that I &lt;em&gt;inject&lt;/em&gt; the flaws myself, so every label is sample-accurate:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Flaw&lt;/th&gt;
&lt;th&gt;How it's injected&lt;/th&gt;
&lt;th&gt;Severity 1 / 2 / 3&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;rushed / dragged&lt;/td&gt;
&lt;td&gt;phase-vocoder time-scale of 4–9 words&lt;/td&gt;
&lt;td&gt;×1.3/1.6/2.0 · ×0.8/0.65/0.5&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;monotone&lt;/td&gt;
&lt;td&gt;WORLD vocoder, pitch contour squashed toward its mean&lt;/td&gt;
&lt;td&gt;50/75/95% removed&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;mumbled&lt;/td&gt;
&lt;td&gt;gain down + low-pass&lt;/td&gt;
&lt;td&gt;−6 dB @ 3 kHz … −16 dB @ 1.1 kHz&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;awkward pause&lt;/td&gt;
&lt;td&gt;room-tone silence inserted mid-phrase&lt;/td&gt;
&lt;td&gt;0.7 / 1.3 / 2.2 s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;missing breath&lt;/td&gt;
&lt;td&gt;a natural pause squeezed out&lt;/td&gt;
&lt;td&gt;50 / 20 / 0% kept&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;stutter&lt;/td&gt;
&lt;td&gt;word onset repeated&lt;/td&gt;
&lt;td&gt;1 / 2 / 3 repeats&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Edits are joined with 5 ms fades that &lt;strong&gt;preserve length&lt;/strong&gt;, so the label times never drift. My first version used 15 ms crossfades, which quietly shifted every later label by 15 ms per edit.&lt;/p&gt;

&lt;p&gt;I also had the same texts read by open &lt;strong&gt;Piper&lt;/strong&gt; TTS voices, with flaws injected into those too. That's the "different speaker" test.&lt;/p&gt;

&lt;p&gt;And I split it honestly: thresholds are tuned &lt;strong&gt;only on JFK&lt;/strong&gt;, then frozen and tested on &lt;strong&gt;Reagan&lt;/strong&gt; plus an unseen voice.&lt;/p&gt;
&lt;h2&gt;
  
  
  Step 3: the bug that wasn't a bug
&lt;/h2&gt;

&lt;p&gt;Version 1 scored each word by how far it departed from the reference, as a robust z-score (median/MAD) after removing your overall offset. On Reagan's own audio with injected flaws it got F1 0.64 with &lt;strong&gt;zero&lt;/strong&gt; false alarms.&lt;/p&gt;

&lt;p&gt;On a TTS voice reading the same text:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;F1&lt;/th&gt;
&lt;th&gt;False alarms on clean audio&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Same speaker&lt;/td&gt;
&lt;td&gt;0.64&lt;/td&gt;
&lt;td&gt;0 / min&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Different speaker&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;0.03&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;41 / min&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;A synthetic voice doesn't breathe where Reagan breathes or lift the words he lifts. Measured against Reagan, &lt;em&gt;everything&lt;/em&gt; it does is a deviation. Technically correct, but useless as feedback: nobody wants to hear that every sentence is wrong because they aren't Reagan.&lt;/p&gt;

&lt;p&gt;The fix was to separate &lt;strong&gt;style&lt;/strong&gt; from &lt;strong&gt;flaws&lt;/strong&gt;. A flaw is something that's off compared with the reference &lt;em&gt;and&lt;/em&gt; off compared with &lt;strong&gt;your own&lt;/strong&gt; delivery. So every word gets two scores:&lt;br&gt;
&lt;/p&gt;
&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;z_ref&lt;/span&gt;  &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;robust_z&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;participant&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="n"&gt;reference&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;   &lt;span class="c1"&gt;# departs from the reference, beyond your overall style
&lt;/span&gt;&lt;span class="n"&gt;z_self&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;robust_z&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;participant_feature&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;       &lt;span class="c1"&gt;# stands out within your own recording
&lt;/span&gt;&lt;span class="n"&gt;flaw&lt;/span&gt;   &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;np&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;fmin&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;z_ref&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;z_self&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;              &lt;span class="c1"&gt;# a soft AND
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;p&gt;A consistently different voice moves &lt;code&gt;z_ref&lt;/code&gt; everywhere but &lt;code&gt;z_self&lt;/code&gt; almost nowhere, so it's reported once as &lt;em&gt;style&lt;/em&gt; ("33% faster and flatter than the reference") instead of 40 times as flaws.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;False alarms on clean different-speaker recordings went from 41/min to 3/min.&lt;/strong&gt;&lt;/p&gt;
&lt;h2&gt;
  
  
  Step 4: a stutter that looked like a pause
&lt;/h2&gt;

&lt;p&gt;Injected stutters kept being reported as "awkward pause". The cause: the aligner usually places the word at its &lt;em&gt;last&lt;/em&gt; restart, so "w- w- we" becomes a long gap followed by a normal "we".&lt;/p&gt;

&lt;p&gt;The fix was one feature: &lt;strong&gt;seconds of voiced audio inside the gap&lt;/strong&gt;. A real pause is silent; a gap full of voiced sound is a restart.&lt;br&gt;
&lt;/p&gt;
&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;gap_voiced&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;np&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;sum&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;~&lt;/span&gt;&lt;span class="n"&gt;np&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;isnan&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f0&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;prev_end&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="n"&gt;word_start&lt;/span&gt;&lt;span class="p"&gt;]))&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="n"&gt;HOP&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;p&gt;Long pauses now require &lt;code&gt;gap_voiced &amp;lt; 0.15 s&lt;/code&gt;, and stutters can trigger on &lt;code&gt;gap_voiced &amp;gt; 0.12 s&lt;/code&gt;. Stutter recall went from 0.11 to 0.44 on the held-out set.&lt;/p&gt;
&lt;h2&gt;
  
  
  Results on the held-out set (Reagan + an unseen voice)
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;detected regions overlap the true flaw with &lt;strong&gt;mean IoU 0.81&lt;/strong&gt;; overall &lt;strong&gt;F1 0.51&lt;/strong&gt;;&lt;/li&gt;
&lt;li&gt;awkward pauses, missing breaths, mumbling and stutters are located within &lt;strong&gt;0.00–0.06 s&lt;/strong&gt;;&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;0 false alarms per minute&lt;/strong&gt; on untouched audio of the reference speaker;&lt;/li&gt;
&lt;li&gt;the rubric score falls monotonically from L0 to L4 on &lt;strong&gt;every&lt;/strong&gt; excerpt (Spearman ρ = −1.00).&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;What doesn't work yet:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Pacing is weakest (F1 0.29–0.35).&lt;/strong&gt; Slowing a phrase also stretches its pauses, so "dragged" is often reported as "awkward pause"; the confusion matrix shows exactly that.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Different speakers are still hard (F1 0.21).&lt;/strong&gt; Fixed noise floors are the next thing to replace with learned ones.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;
  
  
  Three things I'd tell anyone building an evaluator
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Make your labels exact before tuning anything.&lt;/strong&gt; Injecting the flaws yourself turns an argument about subjective judgement into a number.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Hold out a speaker, not just files.&lt;/strong&gt; My calibration F1 (0.62) and held-out F1 (0.51) are close &lt;em&gt;because&lt;/em&gt; the split was honest. Same-speaker numbers alone would have hidden the 41-per-minute problem.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Decide what &lt;em&gt;shouldn't&lt;/em&gt; count before deciding what should.&lt;/strong&gt; The hardest part wasn't detecting deviations, it was deciding which deviations matter.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Code, dataset (207 labelled recordings) and the full evaluation are open source:&lt;/p&gt;


&lt;div class="ltag-github-readme-tag"&gt;
  &lt;div class="readme-overview"&gt;
    &lt;h2&gt;
      &lt;img src="https://assets.dev.to/assets/github-logo-5a155e1f9a670af7944dd5e12375bc76ed542ea80224905ecaf878b9157cdefc.svg" alt="GitHub logo"&gt;
      &lt;a href="https://github.com/JayPokale" rel="noopener noreferrer"&gt;
        JayPokale
      &lt;/a&gt; / &lt;a href="https://github.com/JayPokale/podium" rel="noopener noreferrer"&gt;
        podium
      &lt;/a&gt;
    &lt;/h2&gt;
    &lt;h3&gt;
      Contrastive speech delivery analytics: find exactly where a delivery drifts from a great speech, and why. Multimodal AI Hackathon 2026, Track C.
    &lt;/h3&gt;
  &lt;/div&gt;
  &lt;div class="ltag-github-body"&gt;
    
&lt;div id="readme" class="md"&gt;&lt;div class="markdown-heading"&gt;
&lt;h1 class="heading-element"&gt;🎙️ Podium: contrastive speech delivery analytics with temporal flaw grounding&lt;/h1&gt;
&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;Read a great speech. Podium shows exactly where your delivery drifts from it, to the tenth of a second
and explains each flaw with the numbers behind it.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Built for the Multimodal AI Hackathon 2026, &lt;strong&gt;Track C: Contrastive Speech Analytics &amp;amp; Temporal Flaw Grounding&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;&lt;a rel="noopener noreferrer" href="https://github.com/JayPokale/podium/docs/img/dashboard_full.png"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fraw.githubusercontent.com%2FJayPokale%2Fpodium%2FHEAD%2Fdocs%2Fimg%2Fdashboard_full.png" alt="Podium dashboard: rubric, pitch/loudness/pace overlays with flaw regions, explained flaw cards"&gt;&lt;/a&gt;&lt;/p&gt;
&lt;div class="markdown-heading"&gt;
&lt;h2 class="heading-element"&gt;What it does&lt;/h2&gt;
&lt;/div&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Pick a reference&lt;/strong&gt;: an excerpt of a great public-domain speech (JFK, Reagan).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Give your delivery&lt;/strong&gt;: record yourself reading the same text in the browser, upload a file, or try one of
the 207 dataset samples.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Get grounded feedback&lt;/strong&gt;
&lt;ul&gt;
&lt;li&gt;a deterministic &lt;strong&gt;0-10 rubric&lt;/strong&gt; for pacing, pauses, pitch, volume and fluency;&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;flaw regions&lt;/strong&gt; with exact start/end times, shaded on time-series overlays of &lt;em&gt;your&lt;/em&gt; pitch, loudness and
pace against the reference (time-warped onto your timeline word by word);&lt;/li&gt;
&lt;li&gt;a &lt;strong&gt;causal explanation&lt;/strong&gt; for every region, built from the measured delta ("14.3 syllables/s here…&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ol&gt;&lt;/div&gt;
  &lt;/div&gt;
  &lt;div class="gh-btn-container"&gt;&lt;a class="gh-btn" href="https://github.com/JayPokale/podium" rel="noopener noreferrer"&gt;View on GitHub&lt;/a&gt;&lt;/div&gt;
&lt;/div&gt;


&lt;p&gt;&lt;em&gt;Built for the Multimodal AI Hackathon 2026 (Track C), with heavy help from Claude Code, an AI coding agent, for implementation and evaluation runs.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>python</category>
      <category>machinelearning</category>
      <category>opensource</category>
    </item>
    <item>
      <title>Outside Quest: I asked a local Gemma to name birds. It couldn't, so it sends you on a scavenger hunt instead</title>
      <dc:creator>Jay Pokale</dc:creator>
      <pubDate>Tue, 06 Oct 2026 06:05:08 +0000</pubDate>
      <link>https://dev.to/jaypokale/outside-quest-i-asked-a-local-gemma-to-name-birds-it-couldnt-so-it-sends-you-on-a-scavenger-l2a</link>
      <guid>https://dev.to/jaypokale/outside-quest-i-asked-a-local-gemma-to-name-birds-it-couldnt-so-it-sends-you-on-a-scavenger-l2a</guid>
      <description>&lt;p&gt;&lt;em&gt;This is a submission for the &lt;a href="https://dev.to/challenges/hacktoberfest-week1-2026-10-05"&gt;Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  What I Built
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Outside Quest&lt;/strong&gt; is a photo scavenger hunt that runs on Google's open-weight &lt;strong&gt;Gemma 4&lt;/strong&gt;, entirely on a laptop with Ollama. It's designed so the screen is the shortest part of the experience:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Before the walk (about 2 minutes):&lt;/strong&gt; type where you're going. Gemma writes a six-item quest card that fits the real place and season. Print it, or glance at it once. Then a full-screen &lt;strong&gt;"Phone away 🌳"&lt;/strong&gt; card takes over.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;On the walk (an hour):&lt;/strong&gt; the phone stays in your pocket. You only take it out to photograph a find.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Back home (a few minutes):&lt;/strong&gt; drop in your photos. Gemma checks which quests each photo completes and says what it saw. You get points, badges and a walk journal: a timeline from the photos' timestamps and a small route map drawn from their GPS tags, with no map tiles and no internet.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;It's for anyone who says "I should go outside more" and then doesn't: families with kids, friends who need an excuse for a walk, or me after a day of staring at code.&lt;/p&gt;

&lt;p&gt;The quest card adapts to where you are. For &lt;em&gt;Ambazari lake garden, Nagpur, India&lt;/em&gt; in October it asked for "a seed pod or fruit on the ground" and "moss or damp earth" (post-monsoon tropical). For &lt;em&gt;Central Park, New York&lt;/em&gt; it asked for "crimson leaf under a sturdy oak" and "a squirrel busy gathering nuts". That only happened after I told the prompt to think about the real local climate; the first version cheerfully asked Nagpur for "autumn colour change".&lt;/p&gt;

&lt;h2&gt;
  
  
  Demo
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F5rk493o785hzxn3ml0wt.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F5rk493o785hzxn3ml0wt.jpg" alt="Quest card for Ambazari lake garden, Nagpur, generated locally by Gemma 4" width="799" height="416"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fyrdimvpgpawk4ryl2nb5.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fyrdimvpgpawk4ryl2nb5.jpg" alt="Phone away screen" width="799" height="416"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fq8g17k2jm4xzqsn1cl3k.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fq8g17k2jm4xzqsn1cl3k.jpg" alt="Results board after checking six photos" width="799" height="416"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F9ow8bygrgl8940njbvdp.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F9ow8bygrgl8940njbvdp.jpg" alt="Walk journal with timeline and offline route" width="799" height="416"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;The results above use six Wikimedia Commons photos (credited in the repo) with made-up timestamps and GPS points, so I could test the journal and map.&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Code
&lt;/h2&gt;


&lt;div class="ltag-github-readme-tag"&gt;
  &lt;div class="readme-overview"&gt;
    &lt;h2&gt;
      &lt;img src="https://assets.dev.to/assets/github-logo-5a155e1f9a670af7944dd5e12375bc76ed542ea80224905ecaf878b9157cdefc.svg" alt="GitHub logo"&gt;
      &lt;a href="https://github.com/JayPokale" rel="noopener noreferrer"&gt;
        JayPokale
      &lt;/a&gt; / &lt;a href="https://github.com/JayPokale/outside-quest" rel="noopener noreferrer"&gt;
        outside-quest
      &lt;/a&gt;
    &lt;/h2&gt;
    &lt;h3&gt;
      Two minutes on a screen, an hour outside: an offline photo scavenger hunt powered by Gemma 4 + Ollama. Hacktoberfest 2026 Touch Grass.
    &lt;/h3&gt;
  &lt;/div&gt;
  &lt;div class="ltag-github-body"&gt;
    
&lt;div id="readme" class="md"&gt;&lt;div class="markdown-heading"&gt;
&lt;h1 class="heading-element"&gt;Outside Quest 🌿&lt;/h1&gt;
&lt;/div&gt;
&lt;p&gt;&lt;strong&gt;Two minutes on a screen, an hour outside.&lt;/strong&gt; An offline photo scavenger hunt powered by Google's open-weight
&lt;strong&gt;Gemma 4&lt;/strong&gt;, running on your own computer with &lt;a href="https://ollama.com" rel="nofollow noopener noreferrer"&gt;Ollama&lt;/a&gt;.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Before you go:&lt;/strong&gt; tell it where you're walking. Gemma writes a 6-item quest card that fits the real local
season ("a seed pod on the ground" in tropical Nagpur in October, "crimson leaves under an oak" in New York)
Print it or glance at it once.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;On the walk:&lt;/strong&gt; phone in your pocket. You only take it out to snap a photo of each find.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Back home:&lt;/strong&gt; drop in your photos. Gemma checks which quests each photo completes, with the visible evidence
and builds a walk journal: a timeline from the photos' timestamps and a little route map from their GPS tags.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;No account, no upload, no internet needed. Your photos (and their location data) never leave the computer.&lt;/p&gt;
&lt;p&gt;…&lt;/p&gt;&lt;/div&gt;
  &lt;/div&gt;
  &lt;div class="gh-btn-container"&gt;&lt;a class="gh-btn" href="https://github.com/JayPokale/outside-quest" rel="noopener noreferrer"&gt;View on GitHub&lt;/a&gt;&lt;/div&gt;
&lt;/div&gt;


&lt;h3&gt;
  
  
  Try it
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;ollama pull gemma4:e2b
git clone https://github.com/JayPokale/outside-quest &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="nb"&gt;cd &lt;/span&gt;outside-quest
pip &lt;span class="nb"&gt;install&lt;/span&gt; &lt;span class="nt"&gt;-r&lt;/span&gt; requirements.txt &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; python3 server.py   &lt;span class="c"&gt;# http://127.0.0.1:8777&lt;/span&gt;
&lt;span class="c"&gt;# optional, for durable checking:&lt;/span&gt;
temporal server start-dev &lt;span class="nt"&gt;--db-filename&lt;/span&gt; ~/.outside-quest/temporal.db
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;No account, no API key. There's also a &lt;strong&gt;Quick card&lt;/strong&gt; button that works with no model at all.&lt;/p&gt;

&lt;h2&gt;
  
  
  How I Built It
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Stack:&lt;/strong&gt; &lt;code&gt;gemma4:e2b&lt;/code&gt; through Ollama on a 6 GB GTX 1660 Ti laptop GPU · a Python standard-library server plus Pillow (for EXIF) · an optional local Temporal server for durable checking · one HTML file, no framework. A quest card takes ~30 seconds, checking a photo ~30 seconds.&lt;/p&gt;

&lt;h3&gt;
  
  
  Plan A failed, and that's the most useful thing I learned
&lt;/h3&gt;

&lt;p&gt;My first idea was the obvious one: a bird and plant identifier that works on the trail with no signal. Before building the UI, I tested the model on six photos: a Common Myna, a neem tree, a Monarch butterfly, a fly agaric mushroom, a peacock and a storm cloud.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Photo&lt;/th&gt;
&lt;th&gt;gemma4:e2b said&lt;/th&gt;
&lt;th&gt;gemma4:e4b said&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Common Myna&lt;/td&gt;
&lt;td&gt;"Jackdaw", confidence &lt;strong&gt;high&lt;/strong&gt;
&lt;/td&gt;
&lt;td&gt;"Weaver Bird", confidence &lt;strong&gt;high&lt;/strong&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Monarch butterfly&lt;/td&gt;
&lt;td&gt;"butterfly, undetermined"&lt;/td&gt;
&lt;td&gt;"Heliconius"&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Neem leaves&lt;/td&gt;
&lt;td&gt;"deciduous trees, undetermined"&lt;/td&gt;
&lt;td&gt;"trees / forest canopy"&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Peacock&lt;/td&gt;
&lt;td&gt;Peacock ✓&lt;/td&gt;
&lt;td&gt;Peacock ✓&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Asking for a top-3 list didn't rescue it: the Myna never appeared. Gemma 3 4B did worse and broke its JSON. A tool that names the wrong bird &lt;em&gt;with high confidence&lt;/em&gt; is worse than no tool, especially outdoors, where people make decisions about mushrooms and berries.&lt;/p&gt;

&lt;p&gt;But in every run, the &lt;strong&gt;coarse&lt;/strong&gt; answer was right: &lt;em&gt;there's a bird, a butterfly on a flower, red-capped mushrooms, a storm cloud, a canopy of leaves&lt;/em&gt;. So I flipped the design: the model only answers the question small open models are good at, "does this photo show a bird?", and the naming is left to you and a field guide. A scavenger hunt turned out to be a better way to get people outside anyway.&lt;/p&gt;

&lt;h3&gt;
  
  
  Making the judge strict
&lt;/h3&gt;

&lt;p&gt;The first judge was too generous: it gave "a leaf" points for the butterfly photo because there were leaves in the background. Word-matching heuristics made it worse (it then rejected the peacock for "a bird"). What worked was asking Gemma, through a JSON schema, two explicit yes/no questions for &lt;strong&gt;every&lt;/strong&gt; quest on &lt;strong&gt;every&lt;/strong&gt; photo:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"q2"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"evidence"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"A butterfly (an insect) is clearly resting on the flower."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"is_main_subject"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"completed"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;A quest only counts if both are true. On a test card (a bird, an insect on a flower, a mushroom, a stormy cloud, leaves up close, something red) and the six test photos, that gave &lt;strong&gt;5 correct matches and no false ones&lt;/strong&gt;. The miss was the storm cloud, which it read as a seascape. It isn't perfect: in the run shown above it gave "a leaf bigger than your hand" to a canopy photo, where nobody can judge leaf size. But it errs strict far more often than generous, and for a game that's the right way round.&lt;/p&gt;

&lt;h3&gt;
  
  
  Safety lives in code, not in the prompt
&lt;/h3&gt;

&lt;p&gt;Quests the model writes pass through plain-code filters before anyone sees them. Anything about climbing, entering water, touching, picking, eating, feeding or chasing animals, roads, railway tracks, private land or night walks is dropped and replaced from a safe built-in pool. Mushrooms, berries, nests and eggs always get "(photo only, don't touch)". If the model is down or returns junk, the card fills from the same pool, and there's an instant "Quick card" button that skips the model entirely.&lt;/p&gt;

&lt;h3&gt;
  
  
  Making the photo check durable with Temporal
&lt;/h3&gt;

&lt;p&gt;Checking is the slow part: about 30 seconds per photo, so a 40-photo walk is 20 minutes of a laptop GPU working. The first version looped over photos in the browser. Close the tab, let the laptop sleep, or have Ollama run out of memory on photo 23, and the rest of the batch was simply gone.&lt;/p&gt;

&lt;p&gt;So the Gemma judge now runs inside a &lt;a href="https://temporal.io" rel="noopener noreferrer"&gt;Temporal&lt;/a&gt; workflow, on the open-source Temporal dev server on the same laptop. Each walk is a &lt;code&gt;CheckWalk&lt;/code&gt; workflow and each photo is a &lt;code&gt;check_one&lt;/code&gt; activity:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Photos are saved to disk first, so you can close the tab as soon as the upload finishes. Reopen the page later and it picks the walk back up.&lt;/li&gt;
&lt;li&gt;If Ollama is down, out of memory, or returns broken JSON, the activity is retried with backoff (5 seconds up to 2 minutes, 30 attempts) instead of being skipped.&lt;/li&gt;
&lt;li&gt;If the server dies, the workflow resumes from the next unchecked photo. Each answer is cached next to its photo, so nothing is checked twice.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;To test it, I stopped Ollama in the middle of a six-photo walk and then killed the app server too. The red bar is the photo that kept failing while Ollama was down. Attempt 4 succeeded once it came back, and the walk finished with all six photos checked:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fdx40wq1gqwpzwv5tgbtn.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fdx40wq1gqwpzwv5tgbtn.jpg" alt="Temporal timeline: a photo check retried while Ollama was down, then the walk finished" width="800" height="148"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;If Temporal isn't running, the app works exactly as before. Durable mode is an extra, not a requirement.&lt;/p&gt;

&lt;h3&gt;
  
  
  What a Reddit reader caught
&lt;/h3&gt;

&lt;p&gt;After I shared Outside Quest on Reddit, a reader pointed out something I hadn't tested: WhatsApp strips EXIF data, and iPhone sharing can leave location out. So for many people the route map and timeline would just be empty, with no error. The journal now says so ("no location in 2 of 5 photos") and suggests copying the originals over USB or AirDrop. Fixing that also turned up a bug: the walk length mixed camera times with file dates, which made one test walk "3175 minutes" long. It now uses camera times only.&lt;/p&gt;

&lt;p&gt;They also asked whether the card changes with the season. It does. Same place, Central Park, three months: October asked for "a fallen maple leaf", January for "ice crystals on a puddle", and April for "a daffodil or early tulip bloom".&lt;/p&gt;

&lt;h2&gt;
  
  
  Why Does Open Innovation Matter?
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Your photos are a map of where you were.&lt;/strong&gt; Walk photos carry GPS coordinates and timestamps. Sending them to a cloud API would hand someone a log of when and where you walk. Here the model and the photos stay on your laptop.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;It works where the walk is.&lt;/strong&gt; No API key, no signal needed, no per-photo cost, so a family can check a hundred photos after a holiday without a bill.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;I could test it honestly and change course.&lt;/strong&gt; Because the model runs locally and is free to call, I could benchmark e2b against e4b and Gemma 3 on the same photos in an evening, see exactly where it fails, and design around that instead of trusting a demo.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Where closed would win:&lt;/strong&gt; a frontier cloud model would almost certainly name the Myna. For this app that's the wrong trade: I'd rather have a private, free, offline game that's honest about what it knows.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  My Agent Session
&lt;/h2&gt;

&lt;p&gt;Built with heavy help from Claude Code (an AI coding agent): it ran the model benchmarks above, wrote most of the code, and drove the UI in a browser to test it. The design decisions (dropping species ID, strict judging, code-level safety filters) came out of those test runs.&lt;/p&gt;

&lt;h2&gt;
  
  
  Prize Categories
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Best Use of Gemma&lt;/li&gt;
&lt;li&gt;Best Use of Temporal&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>devchallenge</category>
      <category>hf26challenge</category>
      <category>gemma</category>
      <category>opensource</category>
    </item>
  </channel>
</rss>
