<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Unum</title>
    <description>The latest articles on DEV Community by Unum (@corpunum).</description>
    <link>https://dev.to/corpunum</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4246576%2F7fda5675-3c60-42a9-874f-fa8e9dd981a1.png</url>
      <title>DEV Community: Unum</title>
      <link>https://dev.to/corpunum</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/corpunum"/>
    <language>en</language>
    <item>
      <title>unumsearch: a trigram index so coding agents stop re-reading 258,000 files</title>
      <dc:creator>Unum</dc:creator>
      <pubDate>Sun, 15 Nov 2026 22:39:51 +0000</pubDate>
      <link>https://dev.to/corpunum/unumsearch-a-trigram-index-so-coding-agents-stop-re-reading-258000-files-4mik</link>
      <guid>https://dev.to/corpunum/unumsearch-a-trigram-index-so-coding-agents-stop-re-reading-258000-files-4mik</guid>
      <description>&lt;p&gt;An AI coding agent asks the same sort of question hundreds of times a day: where is this function called, which files mention that setting. A plain &lt;code&gt;rg&lt;/code&gt; answers each one by reading every file again. That is fine on one repository. It gets slow when you run many checkouts on one machine, which is our situation.&lt;/p&gt;

&lt;p&gt;So we built unumsearch, a search CLI for code and files that keeps an index and answers like ripgrep, usually in milliseconds. It is open source (Apache-2.0), written in Rust, and has no GPU, model or network dependency. This post is what it does, what we measured, and where it loses.&lt;/p&gt;

&lt;h2&gt;
  
  
  How it works
&lt;/h2&gt;

&lt;p&gt;The index is a list of every three-letter piece of text and the files it appears in. A query is cut into those pieces, the index names the few files that contain all of them, and only those files are opened and checked line by line against their real contents. The index only narrows what to read; the answer always comes from the file.&lt;/p&gt;

&lt;p&gt;Two properties matter for agents:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;It says when it might be stale.&lt;/strong&gt; A file watcher marks edits, changed files are searched directly until the index catches up, and every JSON answer carries &lt;code&gt;fresh&lt;/code&gt;, &lt;code&gt;backend&lt;/code&gt;, &lt;code&gt;covered&lt;/code&gt;, &lt;code&gt;truncated&lt;/code&gt; and &lt;code&gt;complete&lt;/code&gt; fields so a client knows when to scan for itself.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;It does not index secrets.&lt;/strong&gt; &lt;code&gt;.ssh/&lt;/code&gt;, &lt;code&gt;.aws/&lt;/code&gt;, &lt;code&gt;.env*&lt;/code&gt;, &lt;code&gt;*.key&lt;/code&gt;, &lt;code&gt;*.pem&lt;/code&gt; and similar are never indexed, whatever the configuration says.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  What we measured
&lt;/h2&gt;

&lt;p&gt;The test set is 20 regexes mined from a week of real coding-agent tool calls, on the author's machine, dated 2026-10-09.&lt;/p&gt;

&lt;p&gt;Over the whole tree (247,535 files, warm page cache, median of three passes), ripgrep 15.1 took 347 ms per query. The unumsearch daemon took 64 ms and the one-shot CLI took 91 ms.&lt;/p&gt;

&lt;p&gt;In a separate race of 12 patterns our agents actually issued, over about 180 worktrees (258k files), ripgrep needed 4.03 s in total and the daemon needed 217 ms, 18.6x faster, with the same files as rg on 12 of 12.&lt;/p&gt;

&lt;p&gt;On answers: 19 of 20 queries returned exactly rg's files; the 20th differs only by &lt;code&gt;.env.example&lt;/code&gt; files, which unumsearch never indexes on purpose. During &lt;code&gt;git checkout&lt;/code&gt; of a branch that rewrites 44,686 Linux kernel files, the index caught up 6.4 to 7.5 s after the checkout began. Until then answers said &lt;code&gt;fresh: false&lt;/code&gt;, and 0 of 64,454 fresh answers differed from ripgrep.&lt;/p&gt;

&lt;h2&gt;
  
  
  Where it loses
&lt;/h2&gt;

&lt;p&gt;On a single repository of 3.7k files, ripgrep takes a median 9.8 ms. The daemon takes 2.3 ms, but the one-shot CLI, which starts a new process per query, takes 12 ms. That is slower than rg. The speed comes from a process that keeps the index open, so if you run it as a daemon, MCP server or &lt;code&gt;rpc&lt;/code&gt;, you get the gain; if you shell out once per query on a small repo, you do not.&lt;/p&gt;

&lt;p&gt;The benchmark file lists further weak spots, including that a busier machine made every engine about 1.7x slower in one run, so read the absolute times with that in mind.&lt;/p&gt;

&lt;h2&gt;
  
  
  Try it
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;curl -fsSL https://github.com/corpunum/unumsearch/releases/latest/download/install.sh &lt;span class="o"&gt;|&lt;/span&gt; sh
unumsearch --root ~/src/myrepo index
unumsearch search &lt;span class="s1"&gt;'fn\s+main'&lt;/span&gt; ~/src/myrepo --text
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;For agents, &lt;code&gt;unumsearch mcp&lt;/code&gt; runs it as a stdio MCP server, and the README lists setup for Claude Code, Codex CLI, Gemini CLI and others. The latest release at the time of writing is v0.1.12, which fixed a case where a daemon reported &lt;code&gt;fresh&lt;/code&gt; before it had looked at its roots.&lt;/p&gt;

&lt;p&gt;You can reproduce the race on your own code with &lt;code&gt;python3 bench/race.py --root .&lt;/code&gt;; your numbers will differ.&lt;/p&gt;

&lt;p&gt;Source and benchmark method: &lt;a href="https://github.com/corpunum/unumsearch" rel="noopener noreferrer"&gt;github.com/corpunum/unumsearch&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  How this was written
&lt;/h2&gt;

&lt;p&gt;This article was drafted by an AI model (Claude, from Anthropic) working for CorpUnum Labs, from the project's README, benchmark notes and published project facts. Its figures were checked against those files before posting. The numbers come from those files; none were measured by the model.&lt;/p&gt;

&lt;p&gt;We are CorpUnum Labs, in Athens, building OpenUnum, a local-first runtime for user-owned agents. If you want to see what our agents do on that one mini PC, there is a playable lab here: &lt;a href="https://corpunum.com/play?utm_source=devto&amp;amp;utm_medium=article&amp;amp;utm_campaign=unumsearch" rel="noopener noreferrer"&gt;corpunum.com/play&lt;/a&gt;.&lt;/p&gt;

</description>
      <category>rust</category>
      <category>cli</category>
      <category>ai</category>
      <category>opensource</category>
    </item>
    <item>
      <title>3.6x faster local LLM decoding under load, and the six rows where we lose</title>
      <dc:creator>Unum</dc:creator>
      <pubDate>Sun, 15 Nov 2026 21:33:37 +0000</pubDate>
      <link>https://dev.to/corpunum/36x-faster-local-llm-decoding-under-load-and-the-six-rows-where-we-lose-2gkm</link>
      <guid>https://dev.to/corpunum/36x-faster-local-llm-decoding-under-load-and-the-six-rows-where-we-lose-2gkm</guid>
      <description>&lt;p&gt;We run a small crew of AI agents on one mini PC. Two requests are in flight most of the day, so the number we care about is not how fast a model decodes alone. It is how fast it decodes while another request is loading a long prompt. That is where we built our own inference engine, unum-rt, to do better than the open-source engine we started with. Here is what the measurements say, including the parts that do not flatter us.&lt;/p&gt;

&lt;h2&gt;
  
  
  The headline number
&lt;/h2&gt;

&lt;p&gt;While a second request loads a 16k prompt, our engine decodes at 45.1 tokens per second. The open-source engine decodes at 12.4 tokens per second in the same situation. That is the 3.6x in the title. It is one row of ten, from one run per engine.&lt;/p&gt;

&lt;h2&gt;
  
  
  Where we are ahead, and where we are not
&lt;/h2&gt;

&lt;p&gt;The round 2 tally is three wins for us, one tie ("inside the noise") and six rows where the open-source engine is ahead.&lt;/p&gt;

&lt;p&gt;Our wins:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Decoding while a second request loads a 16k prompt: 45.1 against 12.4 t/s.&lt;/li&gt;
&lt;li&gt;Copy-heavy output (600 tokens quoting a document): 82.8 against 70.7 t/s, median of three runs.&lt;/li&gt;
&lt;li&gt;Decoding 256 tokens at 28k depth with nothing else running: 61.1 against 53.4 t/s. Three runs of the same text gave 61.1, 63.5 and 62.4 for us, and 53.4, 54.2 and 54.4 for the other engine.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The tie: pooled over three different texts, decoding is 52.1 against 49.2 t/s. We lead on two texts and trail on one (43.4 against 45.3), so we do not claim it.&lt;/p&gt;

&lt;p&gt;Our losses are mostly on long cold prompts. On a cold prompt of about 29.5k tokens (six runs), we read 1,001 to 1,021 tokens per second and the open-source engine reads 1,209 to 1,238. First token arrives after 29.1 to 29.5 seconds for us, against 23.8 to 24.6. That is our worst row. At 8.8k tokens we are at 994 against 1,067 t/s. Our graphics-memory use after the bench is also higher: 81.4 GiB, against about 75 GiB for the other engine (that second figure is from round 1 and was not re-measured).&lt;/p&gt;

&lt;p&gt;Our own summary of the trade: the five prompt rows are now at 82 to 97% of the open-source engine's speed. They were at 37 to 82% on the day we switched. A lab that mostly sends one long cold prompt at a time should stay on the open-source engine.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why we still run ours
&lt;/h2&gt;

&lt;p&gt;Our agents are never idle. Because requests overlap, the busy-decoding row matters more to us than the cold-prompt rows. The engine has served the lab's agents since 2026-11-10. In the first production day it handled 188 requests with none failing, with a median decode of 43.2 t/s. That is one lab's workload, not a general claim.&lt;/p&gt;

&lt;h2&gt;
  
  
  Method, hardware and limits
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Hardware: Bosgame M5, AMD Ryzen AI Max+ 395, 128 GB unified memory. Both engines ran two slots at a total context of 131,072 tokens.&lt;/li&gt;
&lt;li&gt;Same model, 4-bit, but not the same file: each engine uses its own 4-bit packing, so small differences may come from that.&lt;/li&gt;
&lt;li&gt;One streaming HTTP client for both engines, same prompts, greedy decoding. Time to first token includes queueing.&lt;/li&gt;
&lt;li&gt;Both engines ran in one window on the night of 2026-11-11, between 02:19 and 03:11.&lt;/li&gt;
&lt;li&gt;Noise is large. Run-to-run noise on this box has been 10 to 27% on identical settings from one day to the next. Within one night, repeats stayed within about 4%. A row only goes to an engine when the gap held across repeats or across both rounds.&lt;/li&gt;
&lt;li&gt;Every change was judged against a bar written down before its run. Changes that missed the bar were dropped even when faster, and a change that altered the generated text was not shipped.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Limits you should weigh: one box, one model family, mostly one run per row, and two different 4-bit files. The engine is private for now, so you cannot rerun our side. The page says the method is complete enough to repeat on your own engine and box, and the raw logs are available on request.&lt;/p&gt;

&lt;h2&gt;
  
  
  AI disclosure
&lt;/h2&gt;

&lt;p&gt;This article was drafted by AI agents working from the published bench page, and every figure above was copied from that page rather than generated. Every figure was checked against the published page before posting. The benchmarks themselves were run and written up by the same AI-agent crew that operates the lab, so treat the page as the lab's own report, not an independent test.&lt;/p&gt;

&lt;h2&gt;
  
  
  Links
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Full tables, the round-by-round history and every kept or dropped change: &lt;a href="https://corpunum.com/bench" rel="noopener noreferrer"&gt;https://corpunum.com/bench&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;Try the lab: &lt;a href="https://corpunum.com/play?utm_source=devto&amp;amp;utm_medium=article&amp;amp;utm_campaign=bench" rel="noopener noreferrer"&gt;corpunum.com/play&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>ai</category>
      <category>llm</category>
      <category>performance</category>
      <category>opensource</category>
    </item>
  </channel>
</rss>
