DEV Community

TECHDEV
TECHDEV

Posted on Originally published at github.com

Your build log is 40,000 lines. Your agent's context window is not.

Your build log is 40,000 lines. Your agent's context window is not.

Every time my build failed, I did the same thing: copy the whole log,
paste it to my agent, and watch it burn half its context on progress
spam before it ever reached the actual error. head -100 cuts the
error at the end. tail -100 cuts the setup at the start. grep -i
error
cuts everything that explains why the error happened.

So I built a tiny tool that sits in a pipe and keeps all three:
logtrim.

What it does

One command shrinks verbose logs and tool output down to a token
budget — keeping the beginning, the end, and every error line, while
collapsing the repetitive middle into one-line markers that say what
was dropped.

npm run build 2>&1 | logtrim --budget 1500
Enter fullscreen mode Exit fullscreen mode

How it trims:

  1. Collapse blank runs — 5 blank lines become 1.
  2. Collapse identical runs — 8 copies of the same warning become 2 copies plus ... [6 identical lines omitted] ....
  3. Collapse similar runs — 12 Processed N/1000 records lines become 2 plus ... [10 similar lines omitted] .... "Similar" means the same shape with only numbers differing. Error lines are never collapsed this way.
  4. Truncate absurdly long lines — a 3 KB single-line JSON blob gets cut to --max-line chars with a note.
  5. If still over budget — keep head + error lines + tail, drop the middle, and mark each gap. Middle lines go first so the start and end survive.
  6. Last resort — if error lines alone exceed the budget, they are truncated to share it. Output never exceeds the budget.

Token counts are estimates (characters / 4). Accurate enough to decide
what to keep, and it keeps the tool dependency-free.

Demo

The repo ships a 62-line build log full of progress spam, repeated
warnings, and one real failure:

node bin/logtrim.js examples/sample.log --stats --budget 300
Enter fullscreen mode Exit fullscreen mode

Result: 62 → 23 lines, 715 → 251 tokens — and the two errors, the
stack trace, and the start of the log all survive. With the default
budget (2000 tokens) the same file goes to 48 lines / 559 tokens,
dropping only the collapsed repeats.

Try it

git clone https://github.com/techdevfun/logtrim.git
cd logtrim
npm link   # makes `logtrim` available everywhere
Enter fullscreen mode Exit fullscreen mode
# Trim a failing build before pasting it to your agent
npm run build 2>&1 | logtrim --budget 1500

# Trim a saved log file, see what was cut
logtrim --budget 800 --stats deploy.log
Enter fullscreen mode Exit fullscreen mode

Zero dependencies, plain Node.js 18+. Tests + CI included.

Repo: https://github.com/techdevfun/logtrim

If you try it on a real log and it drops something it shouldn't have,
open an issue — that's the feedback that shapes what gets built next.

Top comments (0)