I audited expect, a Claude Code plugin that runs AI-driven browser regression tests via Playwright. It scans your git diff, generates a test plan with AI, executes it in a real browser, and reports pass/fail.
The skill itself is a markdown file that teaches Claude how to invoke expect-cli. The CLI is where things get interesting.
🔍 The Skill: Safe
Pure markdown. No hooks, no scripts, no executable code. It only teaches Claude to run expect-cli -m "INSTRUCTION" -y after browser-facing changes. Nothing to worry about here.
⚡ The CLI: Proceed with Caution
Telemetry (not disclosed upfront)
Two telemetry systems run by default, neither mentioned in the README:
-
PostHog: sends machine ID (hashed), project ID, session times, pass/fail counts to
us.i.posthog.comwith a hardcoded API key. Opt-out:NO_TELEMTRY=1(yes, with a typo). -
Axiom: sends OpenTelemetry traces with a hardcoded token to
api.axiom.co. More detailed than PostHog — operation timings, error details, annotations.
Neither sends your code content. But hardcoded tokens in source code aren't great practice.
Cookie extraction (the big one)
The @expect/cookies package reads and decrypts cookies from your local browsers:
-
Chrome: launches headless Chrome with your profile, calls
Network.getAllCookiesvia CDP. Also has a SQLite fallback that directly decrypts the cookie DB (AES-128-CBC / AES-256-GCM). -
Firefox: queries
cookies.sqlitedirectly. - Safari: parses the binary cookie file.
These cookies are injected into the Playwright session so tests run with your real auth. This means the tool has access to all your browser cookies — banking, email, everything. Cookies stay local (not sent to servers), but the AI agent controls the browser they're injected into.
Cookie sync is opt-in per session — the CLI asks for confirmation.
Arbitrary code execution
The Playwright MCP tool accepts arbitrary JavaScript:
const userFunction = new AsyncFunction("page", "context", "browser", "ref", code);
This is eval by another name. The AI agent can execute any code in your Node.js process with full Playwright access. This is inherent to the design — it's a browser automation tool — but combined with cookie injection, the blast radius is significant.
License: not actually MIT
FSL-1.1-MIT — restricts competing commercial use for 2 years, then converts to MIT. Worth knowing before you build on it.
🔒 No Malicious Intent Detected
No backdoors, no data exfiltration beyond the disclosed telemetry, no obfuscated code, no hidden network calls. This is a legitimate tool with legitimate (but powerful) permissions.
📋 TL;DR
| What | Verdict |
|---|---|
| Skill (SKILL.md) | Safe — pure markdown |
| Telemetry | Undisclosed, opt-out has a typo |
| Cookie access | Reads + decrypts all browser cookies |
| Code execution | AI agent runs arbitrary JS |
| License | FSL, not MIT |
| Malicious intent | None found |
Understand what you're granting before you install. The skill is harmless, the CLI is powerful. Disable telemetry (NO_TELEMTRY=1), and know that cookie injection gives the AI authenticated access to your browser sessions.
Top comments (0)