AI coding agents are great at writing browser automation code.
They are terrible at knowing whether it works.
Ask Claude Code for a script that fills a form and scrapes the result, and you get something that looks perfect. Clean functions. Sensible selectors. Then you run it and the selector doesn't exist, the dropdown is empty, or the content never loaded.
If you're:
- building scrapers or internal automation tools,
- using Claude Code (or Cursor, Codex, Copilot) for day-to-day work,
- or tired of pasting raw HTML into a chat window,
This matters.
In this guide I connect Claude Code to Gaffa, a browser automation API, and let the agent build a real automation against a real website. Every command, output and number below comes from an actual run.
The problem: agents write automation code blind
Here is the usual workflow when you ask an AI agent for browser automation:
- The agent guesses the page structure from the URL
- It invents class names that "probably" exist
- You run the script, it fails, you copy the error back into the chat
Repeat three or four times.
The agent is doing its best with zero visibility. It never sees the page. It never sees what JavaScript renders after load. It never sees that a dropdown only fills itself after you pick something in another dropdown.
You end up being the agent's eyes.
The real issue isn't intelligence. It's feedback.
A model that can write a Playwright script can absolutely write a good one.
What it lacks is a browser it can call, look at, and test against.
Give the agent a real browser behind an API, plus instructions on how to use it, and the loop changes completely: look at the page, write the request, run it, check the output, fix, repeat. Without you in the middle.
That is exactly what Gaffa's AI skills and CLI do.
What Gaffa is (and what the skills add)
Gaffa is a REST API that runs real browser sessions for you. You send a URL plus a list of actions, and it returns the results.
Available actions include:
-
click,type,select,scroll,waitto interact with the page -
capture_screenshot,capture_dom,generate_markdownto read it -
parse_tableandparse_jsonto turn it into structured data -
loopto repeat actions, for example through pagination
No SDK and no browser to host. Residential proxies are built in, and choosing their location is a single field (proxy_location).
On top of the API, Gaffa publishes two open-source pieces for AI agents:
-
gaffa-for-ai: four agent skills (
gaffa-authoring,gaffa-find,gaffa-debug,gaffa-support) plus a docs MCP server. - gaffa-cli: a command line tool that installs those skills into Claude Code, Codex, Copilot, Cursor, Antigravity or Pi.
The skills are the interesting part. They don't just describe the API. They tell the agent how to behave: never guess a selector, capture the real page first, always set a time limit, read the key from an environment variable, never print it.
👉 You can create a Gaffa account at gaffa.dev and follow along.
Step 1: Install the Gaffa skills into your project
You don't need to install anything globally. The CLI runs through npx.
Create a project folder and install the skills at project scope:
mkdir quotes-bot && cd quotes-bot
npx @gaffa-dev/cli install --tools=claude-code --scope=project -y
Output:
Installed the gaffa skills (version 0.9.0) into 1 place:
./.claude\skills (project) gaffa-authoring, gaffa-debug, gaffa-find, gaffa-support
For a project install, commit the skills directory and its .gaffa-skills.json to share it.
Registered the gaffa-docs MCP server:
Claude Code ./.mcp.json added
Two things happened:
- Four skills landed in
.claude/skills/ - The Gaffa docs MCP server was registered in
.mcp.json, so Claude can search the documentation directly instead of guessing
Project scope means you can commit .claude/skills/ and your whole team gets the same setup. Use --scope=personal if you want it available in every project. Run without flags and the CLI asks which tools and which scope you want.
Alternative: install as a Claude Code plugin. The skills repo is also a plugin marketplace:
claude plugin marketplace add https://github.com/GaffaAI/gaffa-for-ai
claude plugin install gaffa@gaffa --scope project
Same skills, same MCP server. Pick whichever you prefer.
Step 2: Verify the setup with doctor
doctor only reads. It detects which AI coding tools you have and whether the Gaffa skills are set up in each one:
npx @gaffa-dev/cli doctor
Claude Code installed
config ~\.claude
skills gaffa-authoring, gaffa-debug, gaffa-find, gaffa-support (project: ~\gaffa-work\quotes-bot\.claude\skills)
...
Keep it in mind for later too: doctor tells you when your installed skills are behind the published version, and running install again updates them.
Step 3: Add your API key
The skills read the key from one place only: the GAFFA_API_KEY environment variable.
# macOS / Linux / Git Bash
export GAFFA_API_KEY="your_key_here"
# PowerShell
$env:GAFFA_API_KEY = "your_key_here"
Put it in your shell profile so you don't paste it every session. Never commit it.
Create a token in the API Keys section of the Gaffa dashboard.
Before handing control to Claude, I like to confirm the key and network work with one raw request. This one loads a JavaScript-rendered page and returns it as Markdown:
curl -sS -X POST https://api.gaffa.dev/v1/browser/requests \
-H "X-API-Key: ${GAFFA_API_KEY}" \
-H "Content-Type: application/json" \
-d '{
"url": "https://quotes.toscrape.com/js/",
"proxy_location": "us",
"async": false,
"max_cache_age": 0,
"settings": {
"time_limit": 60000,
"actions": [
{ "type": "generate_markdown", "output_type": "inline" }
]
}
}'
Response (trimmed):
{
"data": {
"id": "brq_VzUDkVAjxNNBDXmkGKyRWPQMiRujoQ",
"state": "completed",
"credit_usage": 2,
"http_status_code": 200,
"running_time": "00:00:05.6373027",
"actions": [
{
"type": "generate_markdown",
"output": "# [Quotes to Scrape](/)\n\n[Login](/login)\n\n“The world as we have created it is a process of our thinking..."
}
]
}
}
5.6 seconds. 2 credits. The quotes on that page only exist after JavaScript runs, and they came back.
The plumbing works. Now the agent takes over.
Step 4: Let Claude Code build a real automation
I picked a task that is annoying on purpose.
The search page at quotes.toscrape.com/search.aspx has two dropdowns: author and tag. It looks simple. It isn't. The tag dropdown is empty until you pick an author, because an old-school ASP.NET postback fills it.
The Tag dropdown before picking an author. Screenshot taken with Gaffa's capture_screenshot.
A blind agent would write select author → select tag → click and fail.
Here is the exact prompt I gave Claude Code inside the quotes-bot folder:
Using Gaffa, build a Python script (quotes_search.py, requests library only)
that opens https://quotes.toscrape.com/search.aspx, picks the author
'Albert Einstein' in the first dropdown, then picks the tag 'inspirational'
in the second dropdown, clicks Search, and returns the matching quotes as
JSON (text, author, tag). Also save a screenshot of the results page.
Test it against the real site and refine it until the output is correct.
No selectors. No hints about the postback. Just the goal.
What Claude did first: reconnaissance, not code
Claude loaded the gaffa-authoring skill on its own. It tried the Gaffa docs MCP server first, but I hadn't approved it yet, so it fell back to fetching the docs pages directly (the skill has that fallback built in). Then it read the action reference bundled with the skill.
One small detail I liked: it checked that GAFFA_API_KEY was set by printing only its length. Never the value.
Rendered from the real session transcript.
Then, instead of writing the script, it sent a Gaffa request with generate_simplified_dom to look at the real page.
The first attempt failed. Claude had added output_type: "inline", which that action doesn't accept, and the API answered with a clear invalid_action_param error. Claude dropped the parameter and retried. 3 credits later, it had the page's DOM.
That's where it caught the trap. In its own words:
"The tag dropdown is empty until an author is selected (ASP.NET postback via
__doPostBack). I need to see what tag options populate after selecting Einstein before writing the script."
It sent a second request (select the author, then capture the DOM) to see the tag options appear. Then a third with the full flow plus a screenshot, which it opened and checked.
Rendered from the real session transcript: the failed first request, the DOM with the empty #tag select, and the moment Claude spots __doPostBack.
Only after that did it write quotes_search.py.
This is the behavior the skill enforces: don't guess the page's markup, capture it first.
Testing and refining
With the request proven, Claude wrote quotes_search.py and kept running it against the live site, fixing small issues along the way, until it returned the right quote.
Then it tried to break its own script. It re-ran the default case and tested other author and tag combinations on its own. One test used a tag that doesn't exist for that author. The select failed and later actions came back action_cancelled. Claude correctly identified that as a bad test input, not a script bug, looked up the real tag list for that author with one more Gaffa request, and re-tested with a valid pair.
Then it cleaned up its test files.
No copy-pasting errors. No "here's what the page looks like." I only read the summary at the end.
The request it ended up with
The core of the final script is this action list:
body = {
"url": "https://quotes.toscrape.com/search.aspx",
"proxy_location": "us",
"async": True,
"max_cache_age": 0,
"settings": {
"time_limit": 45_000,
"actions": [
{"type": "select", "selector": "#author", "values": [{"value": author}]},
# The author's onchange triggers an ASP.NET postback that repopulates
# the (initially empty) tag dropdown, so wait for it before selecting.
{"type": "wait", "selector": "#tag option[value]", "timeout": 10_000},
{"type": "select", "selector": "#tag", "values": [{"value": tag}]},
{"type": "click", "selector": "input[name='submit_button']"},
{
"type": "wait",
"selector": ".results .quote",
"timeout": 10_000,
"continue_on_fail": True, # a genuine zero-result search has no .quote
},
{"type": "capture_dom", "custom_id": "results_dom"},
{"type": "capture_screenshot", "size": "fullscreen", "custom_id": "results_screenshot"},
],
},
}
That wait on #tag option[value] is the line a blind agent never writes. It exists because Claude looked at the page.
The rest of the script submits the request asynchronously, polls GET /v1/browser/requests/{id} until state is completed, downloads the DOM and the screenshot, and parses the quotes with regular expressions. All of it reads the key from GAFFA_API_KEY only.
The result
I ran the final script myself, in a fresh shell:
python quotes_search.py
[
{
"text": "“There are only two ways to live your life. One is as though nothing is a miracle. The other is as though everything is a miracle.”",
"author": "Albert Einstein",
"tag": "inspirational"
}
]
And the screenshot Gaffa captured of the results page:
Both dropdowns are filled, the search ran, and the one matching quote appears.
What it cost
Real numbers from the request history (GET /v1/browser/requests) for everything Claude ran: recon, iterations and validation:
| Metric | Value |
|---|---|
| Gaffa requests (billed) | 10 |
| Credits used | 23 |
| Credits per request | 2 to 3 |
| Time per request | 4 to 10 seconds |
| Claude Code session | ~7 minutes |
Request Logs in the Gaffa dashboard: every POST that started a browser request and every GET that polled its result.
The rejected invalid_action_param request never ran, so it cost nothing. Gaffa bills 1 credit per 30 seconds of browser time, plus residential proxy bandwidth at 750 credits per GB, which is why a light page costs about 2 credits with proxy_location set. The Claude Code side ran inside my regular Claude Pro subscription. No separate API bill.
Why this setup works
After the run, three things stood out.
The skills change behavior, not just knowledge.
Docs tell an agent what the API accepts. The skills tell it how to work: capture before you select, set time_limit explicitly, keep recording off in shipped code, keep the key out of logs. Claude followed every one of those rules without being asked.
The browser closes the loop.
Claude caught the postback, an invalid action parameter and a bad test input because it could run a request and read the real output. That's the difference between "plausible code" and "tested code".
Everything stays in plain REST.
The output is a normal Python script with requests. No vendor SDK, no framework lock-in. You can read it, commit it and run it in CI.
Bonus: the other skills
The install also gave Claude three more skills worth knowing:
-
/gaffa-find: give it a URL and a plain-language goal ("find the CEO's name on this site"). It loops through recon, extraction and validation with a built-in credit cap (50 credits per run by default). -
/gaffa-debug: pass a failing request or abrq_*id and it diagnoses the failure from the request recording and proposes a minimal patch. -
/gaffa-support: when you're stuck, it tries to resolve the issue and then packages a redacted report for Gaffa's support team. It spends no credits.
I'll go deeper on gaffa-find and a full data collection workflow in the next article.
Key takeaways
- One command (
npx @gaffa-dev/cli install) connects Claude Code to Gaffa: four skills plus a docs MCP server, committed with your project.doctorconfirms it worked. - Claude found an ASP.NET postback trap by capturing the real page before writing a selector, then tested its own script against the live site without my help.
- A full build-and-test session cost 23 Gaffa credits.
Final thoughts
Browser automation was never hard to write.
It was hard to verify.
Give your coding agent a real browser and the rules to use it well, and it stops guessing. It starts testing.
👉 Try it yourself: create an account at gaffa.dev, run npx @gaffa-dev/cli install, and give Claude Code a page you've been putting off automating.
Looking for technical content for your company? I can help — LinkedIn · kevinmenesesgonzalez@gmail.com










Top comments (0)