Postman silently installed AI plugins in its own update. I built a one-command diff for your dev-tool trust surface

Postman 12.33.4 shipped an update that silently auto-installed Claude Code and Codex AI plugins, wrote a trusted hash for its own SessionStart hook, and opened a remote MCP connection. Users found out from a bug report, not from the tool: postman-app-support#14370, marked SEVERE, 453 reactions, and the hottest Hacker News story of the week at 1,391 points and 461 comments.

No vendor audits itself. After any dev-tool update, the question nobody answers for you is simple: what did the tool I updated yesterday do differently? I built toolwatch to answer it in one command.

What it does

toolwatch snapshots your dev-tool trust surface, then diffs it across updates. One snapshot, one diff, one exit code.

toolwatch snapshot --out before.json
# ... update your tools ...
toolwatch snapshot --out after.json
toolwatch diff before.json after.json   # exit 1 when something trust-relevant changed

It also has check against a baseline (the CI gate) and watch, which repeats the check until something drifts.

What gets snapshotted:

Collector	What it records
VS Code	extensions (name, version, hash) and MCP servers in settings
Claude Code	plugins, hooks (event, matcher, command hash), MCP servers
Cursor	extensions and MCP servers
Postman	plugin directories under the app data dir
Shell	startup files (.bashrc, .zshrc, .profile), hashed, never copied
Git hooks	.git/hooks of repos passed via --repos (opt-in, default off)
Remote endpoints	scheme and host of every configured MCP endpoint (no credentials, ever)

Each collector reports ok, not-installed, or error and never crashes on a missing tool. A snapshot is one JSON file with a sha256 fingerprint of the whole surface.

The demo: replaying the Postman update

The demo below replays the Postman 12.33.4 shape through the real pipeline (reconstructed fixtures with recorded provenance, not anyone's real config):

And the diff output, captured from the real CLI:

$ toolwatch diff before.json after.json
+ ADDED   claude-hook          claude-code:hook:SessionStart:*
           command postman-ai-bridge --session-start --trusted-hash a1b2c3d4
+ ADDED   claude-mcp-server    claude-code:mcp:postman-remote
           http https://mcp.postman.example
+ ADDED   postman-plugin       postman:plugin:claude-code
           version 1.0.0 · dir .config/Postman/plugins
+ ADDED   postman-plugin       postman:plugin:codex
           version 1.0.0 · dir .config/Postman/plugins
+ ADDED   remote-endpoint      endpoint:https://mcp.postman.example
           via claude-code mcp server "postman-remote" (http)
~ CHANGED vscode-extension     esbenp.prettier-vscode
           version 12.4.0
        -> version 12.5.0

6 change(s): 5 added, 1 changed, 0 removed.

DRIFT: 6 trust-relevant change(s) — review, then re-baseline with 'toolwatch snapshot'.
exit: 1

Two design choices worth noting. A version bump shows as one CHANGED row (old version to new version), not as an add/remove pair, so routine updates stay readable. And by default only additions and changes gate the exit code; removals are reported but exit 0, unless you pass --strict. The idea is that a legitimate update bumping a version should not train you to ignore the gate.

Use it as a CI gate

Commit a baseline.json from a known-good machine, then fail the build when the runner's trust surface drifts:

- run: node dist/cli.js check --baseline baseline.json

Where rivals beat us

I compared against the closest tools before building, and toolwatch loses on their home turf:

• mcp-audit beats toolwatch on MCP depth: tool-poisoning detection, live server inspection, credential-exposure patterns, cross-server toxic flows, SARIF output, and a signed advisory feed. toolwatch records that an MCP server appeared or changed; it does not judge its tool descriptions.
• osquery beats toolwatch on fleet scale and realtime events: a daemon with file, process, and socket streams across thousands of hosts, plus a SIEM for correlation. toolwatch is point-in-time snapshots on one machine.
• Nomos beats toolwatch on verifying agent output: 42 AST gates on what an agent changed in a repo. toolwatch never looks at repo content.
• Every rival with a dashboard beats toolwatch on reporting: v0.1.0 has a terminal table and JSON, no HTML dashboard, no alerting channels.

The wedge: none of them answers "what did the dev tool I updated yesterday do differently" across the whole trust surface in one local command with zero setup. That is the one thing toolwatch does.

Honest limits

• Detection, not protection. toolwatch detects changes in the trust surface. It does not sandbox tools, block installs, or judge whether a plugin or MCP server is malicious. If your only question is "is this MCP server malicious", use mcp-audit. It goes deeper on that leg than toolwatch ever will.
• Point-in-time snapshots, one machine. No daemon, no fleet, no realtime events. A change made and reverted between two snapshots is invisible.
• Config surface only. A tool that changes behaviour without touching any collected config (extension, plugin, hook, MCP entry, endpoint, git hook, shell file) is invisible to toolwatch.
• Linux and macOS paths in v0.1.0. Windows config paths are not covered yet.
• Postman plugin paths are best effort. Sandboxed installs (flatpak, snap) with different layouts may report "not installed", honestly, rather than guessing.
• Snapshots are audit artifacts. They list your extensions, hook commands, and endpoint hosts. Do not commit them to a public repo.

Install

Install from source for now (no published npm package yet):

git clone https://github.com/techdevfun/toolwatch.git
cd toolwatch
npm install
npm run build
node dist/cli.js --help

Node 20 or later. Zero runtime dependencies. TypeScript, strict mode. The test suite is 38 tests (25 unit, 13 end-to-end) plus a live smoke test, and CI runs it on Node 20 and 22.

Roadmap

Windows collector paths, VS Code Insiders paths, alerting channels for watch (email, webhook; currently exit code only), an HTML report for a diff, and an npm package once the maintainer token decision lands.

The code is MIT licensed at github.com/techdevfun/toolwatch. If a tool update has ever surprised you after the fact, snapshot before the next one and see what it actually changed.