DEV Community: Prince Jain

Auditing 153 Claude Code & Codex transcripts for #107757 User-Agent and URL email leaks (and filtering 259 noreply@anthropic.com hits)

Prince Jain — Fri, 06 Nov 2026 05:33:48 +0000

When anthropics/claude-code#107757 and #55743 hit the front page today (where Claude Code injects oauthAccount.emailAddress from ~/.claude.json into context and models copy that email into outbound User-Agent headers or query strings), I wrote a script to audit what my own coding agents had sent over the network across 153 local Claude Code and Codex CLI transcripts (1,928 deduplicated tool calls).

Two false-positive traps that bit my first script

The noreply@anthropic.com trailer inflation (259 synthetic vs 77 real email hits): My first pass ran a standard email regex across ~/.claude/projects/**/*.jsonl and ~/.codex/sessions/**/*.jsonl and returned 336 email occurrences. I initially thought my sessions had leaked hundreds of personal and client addresses. Grouping by domain showed 259 of those 336 matches (77.1%) were noreply@anthropic.com and example.com strings injected by Claude Code's own Co-Authored-By commit trailer prompt. Filtering out synthetic trailer domains left 77 real email occurrences across 8 genuine domains (gmail.com 53, mcxindia.com 11, prince.sh 4, adelev8.com 4, nvidia.com 2, aqr.com 1).
Loopback dev-server checks vs external egress (19 loopback vs 11 external): Extracting HTTP URLs from tool call inputs matched 30 requests, 19 of which (63.3%) were local health checks against 127.0.0.1 and localhost. Separating loopback traffic (EGL014) isolated the 11 real external requests (9 to q.prince.sh, 2 to gemini.google.com).

What actually leaked across 153 sessions

Once synthetic trailers and loopback requests were separated, the audit caught:

2 outbound URL query email leaks (EGL008): Two tool calls passed ?authuser=<REDACTED_EMAIL>@gmail.com directly in outbound URLs.
1 outbound file POST upload (EGL010): A Bash curl command uploaded a local payload file via HTTP POST to an external endpoint.
3 remote SSH MCP tunnels (EGL003): Three MCP servers in ~/.claude.json were configured to pipe tool arguments over ssh root@<REDACTED_IP>.
Active hardware and Statsig telemetry IDs (EGL002): userID, machineID, cachedStatsigGates, and cachedGrowthBookFeatures were active in ~/.claude.json because neither DISABLE_TELEMETRY=1 nor CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1 was set in ~/.claude/settings.json.
153/153 sessions (100%) injected my local macOS username path (/Users/<USER>) into prompt context (EGL005).

Running `egressleak` and blocking leaks in `PreToolUse`

I published the zero-dependency auditor as egressleak (npx egressleak):

npx egressleak

To block outbound email and token leaks deterministically without waiting for upstream CLI patches or trusting prompt instructions, run the emit-hook generator mode in npx egressleak. It outputs a zero-dependency Node.js PreToolUse guardrail script (~/.claude/hooks/egress-guard-hook.mjs) that inspects Bash, WebFetch, and WebSearch tool inputs and exits with code 2 if an outbound request includes a real email address in User-Agent, an ?authuser= / ?email= query parameter, or a secret API token (sk-, ghp_, AIza).

Limitations

egressleak inspects .jsonl transcripts and static config files on disk. If an agent executes a Python or Node script that constructs a destination URL dynamically at runtime without passing the URL or header in the tool call arguments, transcript replay cannot see that packet.
The generated PreToolUse hook inspects tool call arguments (Bash, WebFetch, WebSearch) before execution, not child processes spawned inside an already-approved script file.

GitHub: https://github.com/princejain756/egressleak
npm: https://www.npmjs.com/package/egressleak

Replaying 153 Claude Code & Codex Sessions for Production Blast Radius (and Catching 247 Root SSH Calls)

Prince Jain — Fri, 06 Nov 2026 02:27:17 +0000

An ops engineer on r/devops asked today how to explain to teammates why giving an LLM agent root access to production is dangerous when "they built their own wrapper around it and log what it does."

Logging what an agent executes is post-mortem detection, not containment. When an agent holds an unrestricted ssh root@host shell just to run journalctl or systemctl is-active, there is zero technical boundary between a read-only check and systemctl stop, rm -rf, or DROP TABLE over that same socket.

To see what my own local coding agents had actually executed over the past month, I built agentblast (npx agentblast, github.com/princejain756/agentblast) and ran it across all 153 local sessions on my workstation (126 Claude Code sessions in ~/.claude/projects and 27 Codex CLI sessions in ~/.codex/sessions, 1,122 deduplicated shell commands).

What 153 Real Sessions Revealed

AGENTBLAST v0.1.0 · Agent Production Blast-Radius & Permission Auditor
══════════════════════════════════════════════════════════════════════════════
Target            : ~/.claude (126 sessions) + ~/.codex (27 sessions)
Shell Replay      : 1122 deduplicated commands (1124 raw JSONL rows)
Safety Score      : 0/100 (Grade F) · Blast-Radius Index: 100/100
Root/Remote SSH   : 247 calls (241 read-only probes, 6 remote mutations)

247 ssh root@<REDACTED_IP> calls (ABL007): 241 of those 247 calls (97.6%) were strictly read-only inspection commands (ls -la, tail -n 40, systemctl is-active), while 6 performed remote state mutations (systemctl restart, POST webhooks). Because the agent used root@ instead of a restricted read-only SSH principal, every single one of those 241 log checks carried full root blast radius.
2 plaintext API tokens in ~/.claude/settings.json (ABL001) + 0 permissions.deny rules (ABL003): Both ANTHROPIC_API_KEY and ANTHROPIC_AUTH_TOKEN were stored in plaintext inside the env block of settings.json, and permissions.deny was completely empty.
41 auto-trusted project roots in ~/.codex/config.toml (ABL006) + file credential store (ABL005): Over weeks of opening folders, ~/.codex/config.toml had accumulated 41 [projects."..."].trust_level = "trusted" entries across external drives alongside cli_auth_credentials_store = "file".
25 shell commands touching .env or service credentials (ABL010), 12 daemon commands (ABL012), and 10 agent-config reads (ABL013).

Two Mistakes in My First Pass

My first prototype had two bugs that showed up immediately on real data:

False-positive panic on /tmp rm -rf: My initial regex flagged every rm -rf as CRITICAL. Across 1,122 commands, my agents had run rm -rf twice: rm -rf /tmp/quantiti-head and cd /tmp && rm -rf obskill. Flagging ephemeral /tmp scratch cleanup as a critical incident trains developers to ignore the tool. agentblast now inspects target paths and preceding cd /tmp statements, separating non-/tmp recursive deletions (ABL008, CRITICAL) from /tmp / $TMPDIR scratch cleanups (ABL014, LOW).
Leaking production IPs and token prefixes in audit output: Printing raw evidence lines from ~/.claude/settings.json and ~/.claude/projects/**/*.jsonl exposed real server IPv4 addresses and API key prefixes in terminal screenshots. agentblast now runs redactSensitive() by default, masking public IPv4s as <REDACTED_IP>, API tokens as sk-<REDACTED>, and Base64 hook payloads as <REDACTED_B64>.

Try It Locally

# Audit your local ~/.claude and ~/.codex configs + session transcripts
npx agentblast

GitHub: https://github.com/princejain756/agentblast

Auditing 687 React & Tailwind Files for "Vibecoded" UI Tells (and Catching Two False-Positive Traps)

Prince Jain — Thu, 05 Nov 2026 23:35:06 +0000

When I ran the first version of my static UI linter against OpenCode-App (536 .tsx files on my machine), two things broke immediately:

Code blocks and syntax themes triggered false positives. Scanning raw file strings flagged <pre><code> documentation blocks for sample href="#" tags and terminal ANSI color maps for indigo tokens.
Global weight subtraction punished large repos. Subtracting raw rule weights without normalizing for file count gave a 536-file desktop app a 68/100 (Grade C) when 99.1% of its files had zero findings (only 2 dialog components used indigo utilities and 3 routes had an em-dash).

After masking out comments, <pre>, <code>, and <svg> blocks while preserving exact \n line numbers, and normalizing cosmetic penalties by sqrt(4 / filesScanned), I published uislop (npx uislop) to catch 16 deterministic "vibecoded UI" tells across React, Next.js, Vue, Svelte, Tailwind, and HTML.

npx uislop ./src

Real benchmark across 687 UI files

Target            : W.prince/Princeportfolio (129 UI files) vs Vibecoded SaaS Landing
Files scanned     : 129 UI files · 47 findings across 3/6 categories
Human-Craft Score : 65/100 (Grade C) · Vibecode Index: 35/100
Palette Entropy   : 50/297 chromatic tokens in indigo/violet/purple (16.8%)

REAL CORPUS BENCHMARK (687 UI files across 4 repositories)
  rajesh-portfolio (21 UI files)          100/100 (Grade A) ·  0 findings ·  0.0% indigo
  OpenCode-App (536 UI files)              97/100 (Grade A) ·  6 findings · 13.0% indigo
  W.prince/Princeportfolio (129 UI files)  65/100 (Grade C) · 47 findings · 16.8% indigo
  vibecoded-landing.tsx (1 UI file)         0/100 (Grade F) · 24 findings · 95.2% indigo

What the 16 rules catch

Palette & Gradient Defaulting (UIS001-UIS004): bg-clip-text text-transparent bg-gradient-to-* headings, from-indigo-500 via-purple-500 to-pink-500 gradients, files where indigo/violet/purple exceed 35% of all chromatic Tailwind utilities, and blur-3xl rounded-full opacity-20 background orbs.
Icon & Eyebrow Clichés (UIS005-UIS007): Importing 3+ of Sparkles, Zap, Rocket, Shield, Brain, Wand2 from lucide-react, <Sparkles> + animate-ping hero pills above <h1>, and repeated bg-indigo-500/10 rounded-xl p-3 squircle icon boxes.
Layout Monotony (UIS008-UIS009): Stacked grid-cols-1 md:grid-cols-3 sections and backdrop-blur + hover:-translate-y-1 glass cards.
AI Copywriting (UIS010-UIS011): "Supercharge your", "Unleash the power of", "Built for modern teams", "Next-Generation", and em-dash splices in UI text.
Fake Social Proof (UIS012-UIS014): "Sarah Chen", "Alex Rivera", "Acme Corp", "TechFlow", pravatar.cc, and LLM-memorized Unsplash portrait IDs (photo-1534528741775-53994a69daeb).
Hollow Wiring (UIS015-UIS016): Dead <a href="#"> links and onClick={() => {}} / console.log handlers.

Code and CLI are MIT-licensed at https://github.com/princejain756/uislop (and on npm at https://www.npmjs.com/package/uislop).

Replaying 147 Claude Code & Codex Sessions to Tune autoCompactWindow (and Catching a 3x Token Overcount)

Prince Jain — Thu, 05 Nov 2026 21:39:31 +0000

My first script for checking local Claude Code token usage summed raw assistant lines in ~/.claude/projects/**/*.jsonl and overcounted session 4baee81b by 3.0x (33 JSONL usage rows for 11 actual API turns). Whenever a single turn emits text plus multiple tool_use blocks, Claude Code writes separate streaming records sharing the same message.id, and every record repeats the cumulative prompt token count. Codex CLI has a similar quirk where event_msg (token_count) rows repeat identical info.total_token_usage.total_tokens snapshots.

After deduplicating by message.id across 147 local Claude Code and Codex sessions on my machine (3,197 raw records collapsing to 2,284 real API turns, 255.79M input tokens, 43.9% cache hit rate), two things jumped out:

13.4% of turns drove 47.8% of weighted token cost. Specifically, the 307 turns that ran above 200K context accounted for nearly half of all cache-weighted input token-equivalents (where cache_read_input_tokens is weighted at 0.1x and uncached input at 1.0x), peaking at 467,835 tokens in session 78fd79c6.
Early tool outputs carry massive Context Rent. A 12K-token file read or git diff output on turn 3 of a 45-turn session sits in the prompt window for 42 subsequent turns (12,000 * 42 = 504,000 token-turns of context rent). Worse, when compaction finally triggers, the agent often immediately re-reads the exact same files on the next 3 turns because the summary dropped active file paths.

Replaying local transcripts with `compactsim`

I built compactsim as a zero-dependency Node.js CLI to audit ~/.claude/projects and ~/.codex/sessions offline and run counterfactual compaction simulations against your actual turn history.

Run it directly without installing anything:

npx github:princejain756/compactsim

Here is the real output on my 147-session corpus:

CompactSim v0.1.0 - Context Compaction & Token Waste Simulator
Corpus: 147 sessions (123 Claude Code, 24 Codex) | 2,284 API turns (3,197 raw JSONL usage records, 1.40x raw overcount)

Baseline Token Profile:
- Total input tokens: 255.79M (143.49M uncached @ 1.0x, 112.30M cache-read @ 0.1x, 43.9% cache hit rate)
- Weighted input token-equivalents: 160.79M tok-eq
- Peak session context: 467,835 tokens (session 78fd79c6)
- Turns > 200K context: 307 turns (13.4% of turns driving 47.8% of weighted token cost)
- Turns > 400K context: 59 turns (2.6% of turns driving 15.7% of weighted token cost)
- Mid-session cache-bust spikes (>25K uncached jump): 760 spikes across 147 sessions

Counterfactual Policy Replay:
1. Baseline (recorded transcripts):      160.79M tok-eq |   0.0% saved |  9 compactions |  8.7m wait
2. autoCompactWindow = 400,000 tokens:   138.92M tok-eq | -13.6% saved | 10 compactions |  9.8m wait
3. autoCompactWindow = 250,000 tokens:   114.93M tok-eq | -28.5% saved | 11 compactions | 11.9m wait
4. autoCompactWindow = 150,000 tokens:    89.49M tok-eq | -44.3% saved | 17 compactions | 18.2m wait
5. Sliding tool-output eviction (15t):   136.67M tok-eq | -15.0% saved |  9 compactions |  8.7m wait

How the counterfactual simulator works

When you lower autoCompactWindow (for example from the default window down to 250,000), you do not get free savings:

The turn immediately following compaction pays a 1.0x uncached re-prefill cost on the compacted summary plus system prompt (~22K tokens).
If the compaction summary drops active working context, the agent re-reads files on the next 1 to 3 turns.

compactsim models both the uncached compaction re-prefill penalty and the empirical post-compaction file re-read tax observed in your transcripts. On my corpus, autoCompactWindow = 250,000 hits the sweet spot: it cuts weighted token usage by 28.5% (160.79M down to 114.93M token-equivalents) while adding only 2 extra compactions across 147 sessions.

Code and zero-dep CLI are MIT-licensed at https://github.com/princejain756/compactsim.

Naplog: let the logs wait

Prince Jain — Thu, 22 Oct 2026 19:50:34 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady

What I Built

Naplog turns a pasted batch of log lines into a short morning brief. You paste up to 120 lines, a local open-weight model reads them, and you get back at most five incidents. Every incident links to the exact source lines it claims as evidence, so checking its work takes seconds.

The effort it saves is the 6 a.m. scroll. Anyone who has opened a log file planning to "just skim it" and resurfaced 40 minutes later gets to be a little lazier here: skim five slips, then only read the lines that matter.

Naplog did not exist on October 21. It is two days of work, built inside this challenge window.

Demo

The demo has to run on my machine: the app only binds to 127.0.0.1 and that is the whole point. Here it is working on the bundled 16-line synthetic log:

On that sample, gemma3:4b found all five planted issues (three database connection timeouts, a failed nightly report, a disk-full backup, a rejected auth token) in 127.8 seconds on an M1 Max. Not fast. The model reads slowly so I do not have to read at all.

The repo includes the same sample file and a one-command start if you want to run it on your own logs.

Code

princejain756 / naplog

Local morning briefs from pasted logs via Ollama and Gemma. Every issue links back to its source lines. No cloud inference.

Naplog

Let the logs wait.

Naplog turns a pasted batch of log lines into a short morning brief using a local language model through Ollama. Every issue in the brief links back to the numbered lines it came from, so you can check the evidence before acting. Built for Hacktoberfest 2026, Week 3.

No cloud inference, no accounts, no scheduled jobs. The pasted text never leaves this machine and is held only in memory for the lifetime of the process.

Requirements

Node.js 22 or newer (uses only the standard library)
Ollama running on this machine
The gemma3:4b model pulled locally:
```
ollama pull gemma3:4b
```

Initial model download needs internet access. After that, inference runs entirely offline.

Run

npm start

Then open http://127.0.0.1:3217. The app listens on loopback only.

Test

npm test

Checks cover input preparation, schema validation, the Ollama request contract (via a stubbed fetch), the HTTP surface, and…

View on GitHub

Node 22, standard library only, no dependencies.

How I Built It

All inference runs through Ollama with gemma3:4b, a 3.35 GB open-weight model pulled to local disk. The interesting part is not the prompt. It is the contract around it:

contract.json pins the model tag, system prompt, JSON schema, generation options, and limits in one auditable file.
Before text reaches the model, common credential shapes (password=, token=, api_key=, secret=, Bearer values) are replaced with [REDACTED].
The model answers against a strict schema: at most five incidents, each citing input line numbers. Responses are bounded to 64 KiB and validated; malformed output and failures are never cached.
One digest at a time per server. Concurrent calls get HTTP 429 instead of a queue that pretends to scale.
Successful digests sit in an 8-entry in-memory cache keyed by content hash.

The server checks Host and Origin headers, serves an explicit static allowlist, sends no-store and CSP headers, and writes nothing to disk. No log bodies, no model output, no analytics.

40 automated checks cover masking, schema validation, the Ollama request shape, the HTTP surface, and the single-inference lock. npm test runs all of them without needing the model.

Honest bits from the build. The Ollama already installed on this machine was too old to even pull gemma3:4b (HTTP 412), so the first hour went to upgrading Ollama, not writing inference code. And in the first live run the model described retry attempts in an observation but cited only the timeout lines as evidence. Legal, since those lines exist, but a good reminder of what the evidence links are: they show where a claim points, they do not prove it. That is why the UI stamps "AI draft. Check the source lines before acting." over the output.

Why Does Open Innovation Matter?

Logs are close to the worst thing you can send to a hosted API. They carry hostnames, internal IPs, user IDs, customer data, and the occasional credential somebody forgot to scrub. A closed endpoint would make this tool a data leak with extra steps.

An open-weight model turns the privacy claim from contractual into structural. Inference happens in a process on my laptop, the model still works with the cable pulled, each run costs $0, and the model is swappable: change one string in contract.json, run the same test suite, and the new model has to satisfy the same schema. Closed APIs deprecate, reprice, and get acquired. A 3.35 GB file on disk just sits there.

The Lazy Payoff

Real numbers from this build: 128 seconds of model time replaces the 10 to 15 minutes I would otherwise spend reading a batch like this carefully. Input is capped at 120 nonblank lines on purpose. The use case is the morning batch, not forensics. If you are pasting 5,000 lines, you are doing the job the tool was built to avoid.

Prize Categories

Best Use of Gemma: gemma3:4b performs all inference locally through Ollama.

canimcp: which clients silently break your MCP server's tools

Prince Jain — Fri, 16 Oct 2026 18:54:28 +0000

A tool on r/mcp sat invisible to Claude Code for 17 days. A root oneOf in the input schema. No error on the server, no error on the client. The tool just never appeared in the client's list.

Last week SineFrame published a run of Claude Code and Codex against the same ten MCP servers at pinned versions. Five of the ten features split the clients, and the two never failed on the same feature. A tool argument named ids[]? Claude Code drops the tool silently. A 14 KB input schema? Codex compacts it and loses your required nested fields. 2049 tools on one server? Codex keeps the first 2048.

I wanted that list as a check I could run in CI, so I built canimcp: point it at a saved tools/list response or at your server command, and it reports which client+version breaks on what, each finding carrying its evidence grade and a source link.

$ npx canimcp fixtures/quirky-tools.json --client codex

8 findings . 1 error . 5 warnings . 2 advisories . 10 tools checked

tool: create_payment
  [error] root-composition-adjacent . codex 0.162.0
    root oneOf sits next to "properties" (inputSchema)
    A root oneOf or anyOf placed next to "properties" made Codex lose
    the property names - the call reached the server mangled.
    fix: move the composition inside each property
    src: m3-probes-2026-10 . evidence: probe

Two things the evidence forced me to change in my own design:

I almost shipped probe results as equal to measured ones. The published matrix grades some rows as direct binary probes not yet run under the harness, so the corpus carries evidence grades (measured, probe, historical, field-report) and output labels them. A probe is a lead, not a verdict.
I thought of this as a schema linter. Then the Codex detail sunk in: in code mode the model never sees your JSON Schema at all, only a compacted TypeScript declaration generated from it. This is not a "does your schema validate" tool. It is a "what does each client actually see" corpus. The schema is just the part I can check statically.

What it is not: a live test. It sees the schema, not runtime result shapes or tonight's client build. For real client trials there is M3; canimcp is the fast pass in CI before that.

Corpus PRs need a reproduction and client versions. Cursor, Gemini CLI and friends stay unseeded until someone ships evidence.

github.com/princejain756/canimcp - npx canimcp - zero dependencies, Node 18+.

Which MCP client should get seeded into the corpus next?

I let a frontier model audit my npm package - it found 16 bugs in 400 lines

Prince Jain — Thu, 15 Oct 2026 12:48:43 +0000

I built slopsieve, a zero-dep Node CLI that scores a diff or PR for the hygiene signals reviewers check by hand. Before tagging v0.2.0, I ran an adversarial audit of the full source with a frontier model in high-thinking mode.

The findings were real, not theoretical:

diff -u output (no diff --git header) parsed to zero files. The tool reported a confident, completely false 0/100.
A removed line -- i; inside a hunk (a C decrement) was consumed as a file header and corrupted the path.
--fail-at without a value silently became NaN and the gate never fired.
Any package.json version bump counted as "new dependency for a small diff".
test.each parameterization that kept coverage intact was scored as assertion removal.
JSDoc @param/@returns tagged as commented-out code.
catch {} with no parameter (valid ES2019) escaped detection.
Reordered code was classified as whitespace-only churn because I sorted lines before comparing.

All 16 are fixed, each with a regression case. The tool now scores its own diff 0/100 and correctly lints itself, which is how I found out my test file literally contains strings that look like the slop it tests for (fixed with per-file slopsieve-ignore markers, eslint-disable style).

What shipped alongside the fixes: secrets-in-diff detection with redacted evidence, phantom-import checks against the actual checkout, --format github for Actions annotations, .slopsieverc.json, git range and --staged scoring, generated-file and lockfile exclusion.

The lesson I keep relearning: happy-path parsing is where tools like this quietly lie to you. My parser handled diff --git beautifully and reported a confident clean score on input it did not understand at all.

repo: https://github.com/princejain756/slopsieve

what's the worst silent-failure bug a linter has ever handed you?

slopsieve: a diff linter for the AI slop era

Prince Jain — Thu, 15 Oct 2026 12:12:00 +0000

cURL scrapped its bug bounty after getting buried in AI slop. Godot is drowning in slop PRs. GitHub floated a PR kill switch. the maintainers' complaint is consistent: the code looks fine until you read it.

so i built slopsieve, a zero-dep node CLI that scores a diff or PR for the 12 signals a tired reviewer checks at 1am:

$ npx slopsieve fixtures/slop.patch

SLOP SCORE: 100/100 - ALARM
4 files · +37 -8 lines

FLAGS
  [30] test weakening        tests/cart.test.js:24 it.skip(...)
  [25] debug leftovers       src/cart.js:25 console.log("checking qty", qty)
  [20] empty catch/except    src/utils.js:12 } catch (e) {}
  [20] trivial dependency    package.json:17 adds 'is-odd'
  [15] commented-out code    3 commented-out lines
  [10] narration comments    6 narration comments
  [10] dead exports          'isQuantityOdd' referenced nowhere in this diff
  [ 5] vague util sprawl     src/utils.js created

what it does: parses the unified diff, runs 12 weighted checks (it.skip, assertions removed, console.log/debugger, catch-all deps, exports nothing references, "this function..." comments), prints file and line for every flag. --fail-at makes it a CI gate, --json feeds tooling.

the bug that embarrassed me: my debug-leftovers regex dd\s*\( (laravel dump) matched isQuantityOdd( because the string "dd(" hides inside "Odd(". the tool caught nothing wrong with the fixture and everything wrong with my regex. word boundary fixed it; the test fixture keeps it fixed.

what it is not: an AI detector. it scores diff hygiene signals. a human can write a 100/100 slop diff, an LLM can write a 0/100 clean one. the README lists where the heuristics miss: two-line catch blocks, callers outside the diff, comment patterns in other languages.

github.com/princejain756/slopsieve - npx slopsieve owner/repo#123

what is the one signal you wish your PR queue flagged automatically?

I replayed 7 repos' READMEs in fresh clones. 2 of them fail their own quickstart.

Prince Jain — Wed, 14 Oct 2026 19:15:19 +0000

Your README is a promise nobody tests.

Setup instructions rot silently. You rename a script, add an env var, switch a package manager, and the quickstart keeps promising a path that stopped working weeks ago. Nobody on the team notices, because everyone who would notice already has the repo running. The only people who hit the wall are new users, and they bounce without filing an issue.

I got tired of wondering whether mine still worked, so I built a tool that checks.

What it does

coldstart stages your committed state in a temp dir. No node_modules, no .env, none of the untracked files that make it work on your machine and nowhere else. Then it parses the shell commands out of your README and replays them in order, tracking the working directory and environment between steps.

When a step fails, you get the exit code, the last output lines, and the exact README line that sent a new user there.

Static checks before anything runs

A lot of README bugs are detectable without executing a single command:

npm run start documented but no "start" script in package.json
pnpm lockfile present but the README says npm install
curl | bash installers, flagged because they execute on the reader's machine, not in any sandbox
commands that reference files the repo does not ship and the user cannot provide

What it found on my own repos

7 repos, 2 fail their own quickstart. The worst one documents npm run start and there is no start script anywhere in package.json. A reader following the README hits a wall at step 2 and leaves. That is a star I never got and never knew I lost.

Honest limits

It replays what the README documents, not what your CI does. Server-style commands get started, probed, and killed, so it can only tell you whether the app boots, not whether it works. Placeholder arguments like your-api-key get skipped with a note instead of failing.

Try it

npx coldstart-cli            # current directory
npx coldstart-cli <git-url>  # or straight from GitHub
npx coldstart-cli --docker   # real clean room in a container

Zero dependencies, Node 18+. Repo: https://github.com/princejain756/coldstart

If you run it on yours, I genuinely want to know what breaks. The weirder the README, the better the test case.

The MCP security tools all scan. None of them stop the call.

Prince Jain — Wed, 14 Oct 2026 10:16:46 +0000

I spent a morning looking at MCP security tooling. Cisco has a scanner, there are three or four audit CLIs, all of them read your tool descriptions and report sketchy ones. Useful, but they share a blind spot: nothing sits between the agent and the server when the call actually happens.

The failure mode that keeps showing up (the 159-point HN thread on MCP vulnerabilities is mostly this) is boring: the agent calls read_file with ../../etc/passwd, or fetch on http://169.254.169.254, or a get_env tool that politely returns your AWS keys. A scanner that read the tool description last week does nothing here.

So I built mcp-bouncer. It wraps any stdio MCP server as a JSON-RPC proxy:

tools/list responses get filtered, so unapproved tools are invisible to the agent
path args are normalized (dot segments resolved, separators unified) before allow/deny globs match, which kills traversal and most encoding games
url args refuse private IPs, 169.254.169.254, *.internal, localhost, and any scheme you did not list
per-tool rate limits, a response size cap, call timeouts
every decision appended to a local JSONL audit file, with a sha256 digest of the args rather than the args themselves (args hold secrets; digests still let you correlate)

The embarrassing bug

First working version blocked every single call. The reason: my "contains a null byte" check compared against what was supposed to be a literal NUL and was actually an empty string, and everything includes the empty string. Test suite caught it because I assert the allowed path actually passes, which is the whole point of writing tests that check the happy path and not just the blocks.

Limits, honestly

This is protocol-level enforcement. The wrapped server still runs as your user; a hostile binary can read whatever it wants. For that you want a container or a real sandbox.
Stdio transport only. Remote HTTP MCP servers are not covered yet.
A policy you wrote wrong is still wrong. mcp-bouncer scan gives you a starting draft, not a guarantee.

About 500 lines, zero dependencies, MIT.

repo: https://github.com/princejain756/mcp-bouncer

What is the ugliest thing an MCP server has done in your setup?

update: post-launch hardening

After publishing I ran the whole source through three rounds of security review with a frontier reasoning model. 33 findings, all patched with regression tests: json-rpc batches bypassing policy, non-lifecycle methods forwarded unconditionally, relative paths reaching the audit file, unanchored ipv4-mapped ipv6 checks, deep proto keys in arguments, and a rate-limit map that could be wiped by fake tool names. 0.1.1 is on npm now. The final round came back clean.

Your CLAUDE.md and AGENTS.md disagree and you don't know it

Prince Jain — Wed, 14 Oct 2026 07:29:43 +0000

Every agent tool wants its own config file. CLAUDE.md for Claude Code, AGENTS.md for Codex and 30+ others, .cursor/rules for Cursor, .windsurfrules, GEMINI.md, copilot-instructions.md. You write a rule once, paste it into five places, and six months later the files disagree. One says npm, another says pnpm. One says never push to main, another says always.

There are good sync tools (rulesync) but they ask you to adopt a new source-of-truth format. I wanted a linter, not a migration: audit the files I already have.

So I built ruleherd. npx ruleherd reads every instruction file in a repo and reports:

CONFLICT: contradictions between files (tabs vs spaces, npm vs pnpm, never vs always on shared topics)
DEAD REF: backticked paths and @imports pointing at files that don't exist
DEAD CMD: npm run x where no package.json has an x script
STALE: "react 17" in the rules while package.json installs react 19
DEAD GLOB: a .mdc rule whose globs point at a deleted directory (Cursor silently never loads it)
DUPLICATE: paragraphs pasted across files
BLOAT: single files over ~4k tokens that load into every prompt

Score out of 100. Strict mode exits nonzero for CI. Zero dependencies.

Real output on my own machine (it scans global configs too):

GRADE: B (85/100)
9 instruction files · ~4,757 tokens of agent context
  DUPLICATE ~333-char block in 6 files
  DUPLICATE ~939-char block in 6 files
  DUPLICATE ~705-char block in 6 files
          ~/.claude/CLAUDE.md, ~/.gemini/GEMINI.md,
          ~/.config/devin/AGENTS.md, windsurf global rules,
          ~/.agents/rules/, ~/.cursor/rules/browser-tooling.mdc

I hand-sync one universal rules file into five tool locations. It drifted anyway.

The embarrassing part: my first version flagged @types/node as a dead file reference and pnpm install as a missing npm script. Both are in my own config files. Auditing the auditor is humbling.

Honest limits: contradiction detection is heuristic (keyword pairs and never/always polarity, not semantic understanding). Dead-ref checking is deliberately conservative. A clean grade means "no detectable drift", not "your rules are good".

MIT, zero deps: https://github.com/princejain756/ruleherd

Which of your agent config files do you think is the stale one?

i stopped trusting coding agent demos, so i built a tool that interviews them

Prince Jain — Wed, 14 Oct 2026 06:13:41 +0000

i kept trusting agent demos more than i should have, so i built a small runner that makes coding agents prove it on my own tasks.

the idea is boring on purpose: point it at any repo, define a few tasks in plain json, and it runs each agent CLI on a throwaway copy of the project. after the agent finishes, the verify scripts get restored from the source repo before they run. that part matters. an agent with write access can just edit its own checker and pass. i shipped agent-trials without the restore step, audited it, felt dumb, fixed it.

a real run on a 3-task toy suite looked like this:

                 opus    sonnet    haiku
fix-clamp     33.2s     39.5s     32.6s
divide-guard  22.8s     26.8s     90.5s
slug-test     42.7s     67.5s     45.5s
score          3/3       3/3       3/3

scores and medians move around between runs. it's a smoke test, not a leaderboard. the thing i actually use it for is catching "the cli exited 0 but did nothing" and watching cost drift.

zero dependencies, plain json suites, works offline if your agent cli works offline.

github.com/princejain756/agent-trials

what would you want it to measure that it doesn't?