I kept doing the same manual thing while building features: change some code, reload the page, click through the flow by hand to check it still works, repeat. So I built five46 to do that clicking-around for me.
What it does
You give it a URL and a goal in plain English — "log in and confirm the dashboard loads" — and an LLM (your own OpenAI/Anthropic/Gemini/Groq/Bedrock key) drives a real local Playwright browser toward it, one action at a time.
When it succeeds, that exact run gets written out as a real, standalone Playwright spec you keep — so the manual check you were already going to do becomes a permanent regression test afterward, with no LLM or five46 involved in ever running it again.
npm install -g five46
five46 test http://localhost:3000 --goal "log in and confirm the dashboard loads"
Why I built it this way
Fully local, BYOK
Everything runs on your machine.
The only thing that leaves it is the page's visible text sent to your LLM provider per step — always disclosed, never hidden. No cloud sandbox sees your app's traffic or screenshots, which matters if your organization can't send application data off-machine.
Honest failure reporting
A failed assertion comes back as a real finding:
- Screenshot
- DOM snapshot
- LLM-generated root-cause hypothesis
These are clearly separated from tooling failures such as:
- A stuck or looping agent
- An unparseable LLM response
I spent a lot of this project's time specifically hunting for ways it could silently claim success when it hadn't:
- A compound goal only half-verified
- A weak assertion passing before the real check
- One scenario's crash silently hiding another scenario's passing result in batch mode
Speed
LLM round-trip latency dominates wall-clock time, not five46's own code.
Structured planning (enabled by default) performs one upfront LLM call to plan the entire goal, then most execution steps happen without additional live decision-making.
I actually measured this instead of guessing. Using the same login flow against a live demo site with the free Gemini tier:
| Mode | Time |
|---|---|
| Default (structured planning) | ~7–8 seconds |
| Fully adaptive fallback | ~11 seconds |
Other things it does
-
API/backend testing — No browser required. Drives real HTTP requests and generates a plain
node:testscript instead of a Playwright spec. -
MCP server —
five46 mcpexposes five46 as a tool that IDE coding agents (Claude Code, Cursor, etc.) can call directly. - Self-healing selectors — A stale selector gets one bounded, fully disclosed recovery attempt instead of immediately failing the step.
- Story mode — Splits a multi-acceptance-criteria user story into independent goals, executes them concurrently, and reports pass/fail for each acceptance criterion.
Status
This is still an early project.
It's been tested end-to-end with real LLM API keys across dozens of live websites and APIs, but I'd genuinely appreciate feedback—especially on places where "the AI decided this passed" still feels too unproven to trust.
Top comments (0)