DEV Community

Cover image for Choosing what the model sees: a repo-to-prompt tool for 2026
Artur Daschevici
Artur Daschevici

Posted on • Originally published at unicow.dev

Choosing what the model sees: a repo-to-prompt tool for 2026

For about a week I told people my tool packed the Next.js monorepo in 7 to 10 seconds. The real number was 1.6.

I was reading the wrong line of the benchmark output. time gives you user time and wall time, and I'd been quoting user time, which is CPU-seconds summed across cores. My tool runs on five cores. So I was underselling it by 5x, confidently, to anyone who asked. If you take one thing from this post: wall clock is the number humans experience. Quote that one.

Anyway. This is a post about a repo-to-prompt tool. Yes, in 2026. Stay with me.

The awkward timing question first

Repo-to-prompt tools had their moment two years ago, when the workflow was "assemble context by hand, paste it into a chat window, pray." Then agents learned to grep. Claude Code and Cursor pick their own context now, and most people stopped running a packer before every session.

So why build one?

Because sometimes the agent's retrieval is exactly what I don't want. Code review across a whole subsystem. A migration plan. A raw API call where nobody is doing retrieval for me. In those moments I want to pick the files myself, see what it costs in tokens, and know that .env.production didn't tag along.

Three requirements. Fast. Interactive with visible token costs. Secret-scanned.

No tool had all three. The fast ones (yek, repomix-rs) are fire-and-forget: point, dump, done. The interactive one (code2prompt) doesn't show token costs. So I forked code2prompt and built gnaw.

The benchmark

Vendor benchmarks are junk by default. Different machines, cherry-picked flags, and my favorite trick: comparing against output nobody checked. A tool that emits half the files in half the time isn't fast. It's incomplete.

So here's the setup, built to survive you rerunning it:

  • Docker, --cpus 8 --memory 8g. One command, same environment as mine.
  • Corpus: vercel/next.js at 70e3934.
  • hyperfine, 10 runs, warmup, mean ± σ.
  • File-count check: every tool must emit within ~1% of the others or the comparison is void. All landed between 26.6k and 26.9k files.
  • repomix-rs built from source at 5798dc0, not the npm package. The npm build is roughly 2x slower. I benchmarked their engine, not their packaging. If anything this makes my numbers look worse. Good.
  • Scan settings, stated exactly because it matters: three of the five tools secret-scan (gnaw, repomix, repomix-rs). Run 1 turns scanning off where it can be turned off. repomix-rs scans by default and I found no way to disable it, so its run-1 number includes its scan. Which means run 1 is not a clean gnaw-vs-repomix-rs comparison, and I won't quote it as one. Run 2 is the clean one: everything that can scan, scanning.
  • I ran this three times across three days while working on the tool. Absolute times wobbled about ±6% with the host's mood. The ratios barely moved. Trust the ratios; treat the absolutes as "this hardware, that day."

Script and Dockerfile are in the repo. Rerun it. I mean it. My least favorite genre of reply is "works differently on my machine" and I'd rather get it as a GitHub issue with numbers attached.

Run 1: extraction, scanning off where possible

tool mean ± σ peak RSS cpu× files version
gnaw 1.55 ± 0.10 s 378 MB 5.2 26,706 local
repomix-rs* 1.73 ± 0.11 s 311 MB 4.8 26,635 git:5798dc0
code2prompt 2.11 ± 0.22 s 463 MB 4.1 26,706 4.3.0
yek 8.05 ± 0.20 s 228 MB 1.0 26,790 git:0.25.5
repomix 35.44 ± 1.87 s 769 MB 1.1 26,928 repomix@1.16.0, incl. Node startup

*includes its default regex scan, which can't be disabled. See run 2 for the equal-footing comparison.

Run 2: scanning on, for the three tools that can

tool mean ± σ peak RSS cpu× scanner
gnaw 1.76 ± 0.13 s 477 MB 5.2 gitleaks ruleset
repomix-rs 1.78 ± 0.03 s 311 MB 5.2 custom regex set
repomix 35.97 ± 1.24 s 878 MB 1.1 Secretlint, incl. Node startup

What I'll claim, and what I won't

Against the tools people actually have installed, the gap is not subtle: 5x faster than yek. 20x faster than repomix, with both tools scanning — and that repomix number already includes Node startup in both time and memory, and it's running Secretlint, a different ruleset than gnaw's gitleaks. All three caveats, same sentence, because a 20x headline with the fine print three scrolls away is how vendor benchmarks lie. And no, repomix is not the 22-minute dinosaur from the old yek benchmark everyone still quotes. They shipped serious parallelization work. It's 35 seconds now, current version. Still 20x.

Against repomix-rs, with both tools scanning: 1.01 ± 0.08. A dead heat. I'm not going to torture that into a win. Here's the full trade in one breath: gnaw ties repomix-rs on wall-clock, carries 166 MB more peak RSS, and is running the full gitleaks ruleset where repomix-rs runs a custom regex set. The memory is the price of the bigger ruleset. Whether thoroughness is worth 166 MB is your call; I've made mine.

One more honest one. Look at yek's user time: 8.0 CPU-seconds. Now look at gnaw's: 7.4. Nearly the same total work. gnaw spreads it across roughly five of the eight cores while yek sits on one — the cpu× column shows 5.2, and that's what the wall-clock gap is made of. On a 2-core CI runner, expect much less of it. (Why 5.2 and not 8? Traversal is partly I/O-bound. Profiling where the other cores went is on the list.)

Now the number I actually care about. gnaw with scanning off: 1.55s. With a full gitleaks pass over 26,706 files: 1.76s.

The scan costs about 200 milliseconds.

I picked gitleaks knowing it was the heavier, slower option, and braced for a real cost. The bill came to a blink. Whatever era "secret scanning is too slow to leave on by default" belonged to, it's over.

The fork

code2prompt got the big thing right: packing a repo is something you should be able to steer, not just trigger. It has a TUI. Its extraction speed hangs with the native tools. That's why I forked it instead of starting from zero.

What I changed:

Live token counts. The reason gnaw exists. You browse the tree, toggle files and directories in and out, and the token total updates as you go. Deselect test/, watch tokens fall off the bill. A context window is a budget. The tool should show you spending it.

Search that behaves the way I kept expecting it to. Not a bug report, a taste difference. code2prompt's search does what it does; my fingers disagreed with it several times a day, so gnaw's does what my fingers expect. I'll write up the specific UX splits in a follow-up, because they're small, concrete, and the fun kind of arguable.

gitleaks as the default scanner. The thorough option. I assumed I was trading speed for it. See above: 200ms. Easiest trade I've made this year. I'm building a planted-secrets corpus to measure recall and false positives against the lighter scanners properly. Until that's published, "more thorough" is my working belief, not a measured verdict.

A parallel extraction engine. cpu× 5.2. The table already made this argument.

The pure taste changes stay in the fork. Anything that turns out to be an objective fix goes upstream, where it belongs.

What gnaw is not

An agent. If Claude Code is already picking good context for your task, close this tab, you're fine. gnaw is for when you want to be the retrieval step: deterministic, inspectable, scanned, and done in under two seconds instead of twenty minutes of clicking through GitHub copying files like it's 2023.

It's also not done. Next on the roadmap: an MCP server, so agents can call gnaw as a token-budgeted, secret-scanned context extractor instead of this being a manual-only tool. That's the version that makes sense in 2026, and it's the part I'm most looking forward to building.

Check my work

Repo: gnaw. Quickstart in the README, benchmark script and Dockerfile next to it. docker run it, and if your numbers disagree with mine, open an issue. I once misread my own benchmark by 5x; I have no standing to be precious about corrections.

If it's useful, a star helps the next person find it.


Top comments (0)