DEV Community: Aswani Kumar

From Bug Found to Bug Filed: A Bug-Reporter Skill for Claude Code

Aswani Kumar — Sat, 25 Jul 2026 16:35:00 +0000

Part 8 of the "Automating Playwright with Claude Code" series. Our pack from Part 6 catches problems (flaky tests, locator issues); this post adds a Skill that turns a caught problem into a properly filed bug report, using the guardrail patterns from Part 7.

Every Skill we've built so far in this series ends with Claude telling you something's wrong. This post closes that loop: a Skill that takes a failure Claude just found and turns it into a bug report good enough to actually file — with reproduction steps grounded in evidence, not guesswork, thanks to the guardrails from Part 7.

Why a Bug-Reporter Skill Is Worth Building

The gap between "found" and "filed" is where good bugs get lost. A tester who has to manually write up repro steps after already diagnosing the issue often does a rushed job of it, or skips it entirely.
Consistency helps triage. A bug report with the same structure every time is faster for a team to triage than one written differently by every reporter.
It reuses everything you already have. This Skill leans directly on evidence gathered by the flaky-test-debugger Skill from Part 6 — nothing here is written from scratch.
Guardrails matter even more here. A fabricated repro step in a filed bug wastes a developer's time investigating something that isn't real.

Prerequisites

Completed Part 6 (the Skill pack) and Part 7 (guardrails) of this series.
A bug tracker your team actually uses (this post uses GitHub Issues as the example; the same pattern applies to Jira or anything else).

What a Good Bug Report Actually Needs
Building the Bug-Reporter Skill
Applying Guardrails from Part 7
Step-by-Step: From a Flaky Test Finding to a Filed Issue
Customizing the Template for Your Tracker
Conclusion

Step 1: What a Good Bug Report Actually Needs

Before writing the Skill, it's worth being explicit about the fields a useful bug report has — this becomes the Skill's output template:

Title — specific and searchable, not "Login broken."
Steps to reproduce — numbered, exact, based on what was actually observed.
Expected vs actual result — stated separately, not blended into one sentence.
Evidence — a snapshot, trace, screenshot, or log line, not a description alone.
Environment — browser, viewport, environment (staging/prod), and test run ID if available.
Severity/priority suggestion — a starting point for triage, clearly marked as a suggestion.

Step 2: Building the Bug-Reporter Skill

---
name: playwright-bug-reporter
description: "Turn a test failure or bug found during Playwright testing into"
  a filed bug report. Use whenever the user says a bug was found, asks to
  file an issue, or wants a test failure written up as a report.
---

# Playwright bug reporter

## Process
1. Gather evidence first: the relevant snapshot, trace, or log from the
   failure — never write a report from memory of "what probably happened."
2. Fill out the report using this exact structure:

   **Title**: <component/flow> — <specific symptom>, not a vague summary

   **Steps to Reproduce**:
   1. ...
   2. ...
   3. ...

   **Expected Result**: ...

   **Actual Result**: ...

   **Evidence**: <snapshot excerpt, trace line, or screenshot reference>

   **Environment**: <browser, viewport, env (staging/prod), test run ID>

   **Suggested Severity**: <Critical/High/Medium/Low> — flagged as a
   suggestion for the team to confirm, not a final call.

3. Every reproduction step must come from an action that was actually
   taken and observed during testing — not inferred or assumed.
4. Do not suggest a root cause unless directly asked; a bug report
   documents what happened, not why (that's the flaky-test-debugger
   Skill's job from Part 6).

Step 3: Applying Guardrails from Part 7

This Skill is a perfect candidate for the guardrail patterns from Part 7, since a fabricated bug report is actively harmful — it wastes a developer's time chasing something that isn't real:

## Guardrails
- Never include a reproduction step you did not personally execute and
  observe during this session.
- If evidence for any field is missing (no trace, no screenshot), say
  so explicitly in that field rather than leaving it blank or guessing.
- Do not file the issue automatically — always show the report to the
  user for confirmation first, since only they know if it duplicates
  an existing ticket.

That last line matters: filing directly into a tracker is a good candidate for a human-review gate, not full automation — someone still needs to check for duplicates and confirm it's worth filing.

Step 4: From a Flaky Test Finding to a Filed Issue

Here's the whole loop, start to finish:

This checkout test is failing intermittently — can you check it and file a bug if it's real?

playwright-flaky-test-debugger (Part 6) fires first, gathering trace evidence and identifying that the "Place Order" button is being clicked before the cart total finishes updating.
Once a real, reproducible issue is confirmed (not just occasional CI flakiness), playwright-bug-reporter fires next, producing:

**Title**: Checkout — "Place Order" clickable before cart total finishes updating

**Steps to Reproduce**:
1. Add two items to cart on the checkout page.
2. Immediately click "Place Order" without waiting for the total to refresh.
3. Observe the order is placed with the pre-update total.

**Expected Result**: "Place Order" should be disabled until the cart total
finishes recalculating.

**Actual Result**: Button remains clickable during the recalculation window,
allowing an order to be placed with a stale total.

**Evidence**: Trace shows `cart-total` element still has `aria-busy="true"`
at the moment of the click event (trace timestamp 00:04.211).

**Environment**: Chromium, 1280x720, staging, run #4821

**Suggested Severity**: High — flagged for team confirmation.

Claude shows you this report and waits for confirmation before filing — per the guardrail in Step 3.

Step 5: Customizing the Template for Your Tracker

The template above is tracker-agnostic on purpose. To adapt it:

GitHub Issues: map directly to a .github/ISSUE_TEMPLATE/bug_report.md structure, so the Skill's output can be pasted straight in.
Jira: add a step mapping "Suggested Severity" to your project's actual priority field values, since these vary by team.
Any tracker with required custom fields: list them explicitly in the Skill's ## Process section so nothing gets skipped.

Conclusion

This Skill closes the loop between finding a problem and doing something useful with it — and because it inherits the guardrails from Part 7, the reports it produces are grounded in real evidence rather than a plausible-sounding guess. Combined with Part 6's pack, you now have Skills that catch issues, diagnose them, and write them up in a consistent, reviewable format. Next up, we'll zoom out from individual Skills and map how a set like this covers the full Software Testing Life Cycle.

Would your team's tracker need extra fields beyond this template? Let me know what's missing in the comments!

Adding Guardrails to Your Skills: Human-Review Gates and "Never Fabricate" Rules

Aswani Kumar — Fri, 24 Jul 2026 16:30:00 +0000

Part 7 of the "Automating Playwright with Claude Code" series. In Part 6 we built a three-Skill pack — locator policy, page object generator, and flaky test debugger. This post adds the guardrails that keep all three trustworthy, so nobody on your team ships a fabricated result unchecked.

Our flaky-test-debugger Skill from Part 6 already had one line of guardrail language buried in it: "don't report a guess as a finding." This post makes that idea explicit, systematic, and applied across the whole pack — because the biggest risk with AI-assisted testing isn't that Claude gets something wrong occasionally. It's that a wrong result looks exactly as confident as a right one, unless you've built in a way to catch it.

Why Guardrails Matter for Test Automation Skills

A fabricated "pass" is worse than no test at all. A test that silently reports success without actually verifying the right thing gives your team false confidence.
AI agents don't naturally say "I'm not sure." Without explicit instruction, a Skill will often produce its best guess and present it as fact.
Human-review gates aren't a lack of trust — they're where trust is earned. The goal isn't to slow Claude down everywhere; it's to pause at the handful of points where a wrong call is expensive.
This is what separates a hobby Skill from a production one. Any Skill your whole team relies on needs to fail loudly, not silently.

Prerequisites
Completed Part 6 of this series (the three-Skill pack).
Comfortable editing existing SKILL.md files rather than just writing new ones.

Table of Contents

Three Failure Modes Worth Guarding Against
Step 1: The "Never Fabricate" Rule
Step 2: Evidence-Required Reporting
Step 3: Human-Review Gates for Destructive or Ambiguous Actions
Step-by-Step: Retrofitting Guardrails into the Part 6 Pack
Testing That a Guardrail Actually Holds
Conclusion

Three Failure Modes Worth Guarding Against

Before writing guardrail language, it helps to know exactly what you're guarding against:

Failure mode	Example	Why it's dangerous
Fabricated result	Reporting "test passed" without actually re-checking the page state	False confidence in a broken feature
Invented evidence	Describing a root cause without having looked at the actual trace/log	A confident-sounding wrong diagnosis wastes more time than no diagnosis
Silent scope creep	Rewriting a whole test file when asked to fix one assertion	Unreviewed changes ship without anyone noticing the extra edits

Step 1: The "Never Fabricate" Rule

Every Skill in your pack should have an explicit instruction that it's better to say "I couldn't verify this" than to guess.

## Guardrails
- Never report a test result you have not actually observed via a
  fresh `snapshot` or trace. If a snapshot wasn't taken after the final
  action, take one before reporting pass/fail — don't infer the outcome.
- If you cannot determine something with confidence (root cause,
  selector reliability, expected behavior), say so explicitly rather
  than presenting a guess as a finding.

This single block, added consistently across every Skill in a pack, closes most of the "confident but wrong" risk on its own.

Step 2: Evidence-Required Reporting

Beyond just "don't guess," it helps to require that every claim point at something concrete:

## Reporting format
- Every pass/fail claim must cite what was checked (e.g. "confirmed via
  snapshot: success banner text matches 'Account created'").
- Every root-cause claim must cite the specific trace line, log entry,
  or code line that supports it.
- If evidence is missing or inconclusive, report that directly instead
  of filling the gap with a plausible-sounding explanation.

This turns a vague instruction ("be careful") into something Claude can actually check itself against before responding.

Step 3: Human-Review Gates for Destructive or Ambiguous Actions

Some actions are cheap to redo if wrong. Others aren't — deleting a test file, force-pushing a fix, or modifying a shared fixture used by dozens of other tests. Guardrails should explicitly name the moments to stop and ask:

## Stop and ask before...
- Deleting or overwriting an existing test file, rather than adding
  a new one.
- Modifying a shared fixture, helper, or Page Object used by more than
  one test file.
- Marking a test as "known flaky" and skipping it, rather than fixing
  the underlying cause.
- Proceeding past a CAPTCHA, OTP, or other manual-verification step.

Step-by-Step: Retrofitting Guardrails into the Part 6 Pack

Let's apply this to the playwright-flaky-test-debugger Skill from Part 6, which already had a hint of this in step 3 of its process.

Before (Part 6 version):

3. Report the most likely cause with the specific line of evidence from
   the trace/log that supports it — don't report a guess as a finding.
4. Suggest one fix at a time; don't rewrite the whole test speculatively.

After (with explicit guardrail section added):

## Guardrails
- Never report a root cause without a specific trace or log line to
  back it up. If no trace/log is available, say so and ask for one
  rather than speculating.
- Suggest one fix at a time. Never rewrite the whole test file
  speculatively — that hides the actual change being reviewed.
- Stop and ask before modifying any fixture or helper shared by other
  tests, since a "fix" there can silently affect unrelated suites.

Apply the same pattern to playwright-locator-policy (stop and ask before replacing a locator used across many tests) and playwright-page-object-generator (stop and ask before overwriting an existing Page Object file rather than creating a new one).

Testing That a Guardrail Actually Holds

Guardrails are only useful if they actually change behavior under pressure. Test them deliberately:

This test keeps failing randomly, just fix it however you think is best.

A Skill without guardrails might rewrite the whole test speculatively.
A Skill with the guardrails above should instead ask for the trace/log first, cite specific evidence, and propose one targeted fix — pausing before touching any shared fixture. If it doesn't pause where you expect, the guardrail language likely needs to be more specific and closer to the exact situation, rather than a general "be careful" instruction.

Conclusion

A Skill pack without guardrails can still be useful — but a Skill pack with them is one you can actually hand to a whole team without babysitting every result. The "never fabricate," evidence-required, and stop-and-ask patterns in this post are simple to add and dramatically change how much you can trust what comes back. That wraps up the core mechanics of this series — from setup, to tool choice, to token efficiency, to building and hardening a real Skill pack.

Have you hit a case where a Skill (or any AI tool) confidently gave you a wrong result? How did you catch it? Let me know in the comments!

Beyond the Form Tester: Building a Skill Pack for Your Whole Playwright Suite

Aswani Kumar — Thu, 23 Jul 2026 16:30:00 +0000

Part 6 of the "Automating Playwright with Claude Code" series. In Part 4 we built one Skill; in Part 5 we learned why installing many Skills doesn't bloat every session. This post uses both to grow a single Skill into a small, organized **pack* covering more of your actual test suite.*

One Skill is useful. A handful of related Skills, organized as a pack, is what actually replaces a chunk of your team's onboarding docs. This post takes the playwright-form-tester Skill from Part 4 and grows it into a three-Skill pack — a locator policy, a page object generator, and a flaky-test debugger — using the progressive disclosure model from Part 5 so none of it costs you tokens you're not using.

Why Build a Skill Pack Instead of One Big Skill

Single responsibility, same as good code. One Skill per concern is easier to trigger correctly than one giant Skill trying to cover everything.
Independent triggering. A locator-policy Skill and a flaky-test-debugger Skill fire on completely different requests — bundling them into one file would mean loading irrelevant instructions half the time.
Shared references, not duplicated ones. A pack can share a single references/ folder across Skills instead of repeating the same conventions in three separate files.
Grows with your team. New teammates get your locator conventions, your page object style, and your flaky-test triage process automatically — the same idea the wider "Skill pack" ecosystem (directories like qaskills.sh) is built around.

Prerequisites
Completed Part 4 (the playwright-form-tester Skill) and Part 5 (progressive disclosure) of this series.
Comfortable creating multiple sibling folders under .claude/skills/.

Planning the Pack
Skill 1: Locator Policy
Skill 2: Page Object Generator
Skill 3: Flaky Test Debugger
Sharing a Common References Folder
Verifying the Whole Pack
Conclusion

Step 1: Planning the Pack

Before writing anything, it helps to map out which Skill owns which trigger, so descriptions don't overlap:

Skill	Fires on requests like...
`playwright-form-tester` (Part 4)	"test the login form", "validate checkout"
`playwright-locator-policy`	"add a locator for...", "find the best selector for..."
`playwright-page-object-generator`	"generate a page object for...", "scaffold a POM for..."
`playwright-flaky-test-debugger`	"this test is flaky", "why does this fail intermittently"

.claude/skills/
├── playwright-form-tester/
├── playwright-locator-policy/
├── playwright-page-object-generator/
├── playwright-flaky-test-debugger/
└── playwright-shared/
    └── references/

Step 2: Skill 1 — Locator Policy

This Skill enforces a consistent locator strategy instead of letting Claude default to fragile CSS selectors.

---
name: playwright-locator-policy
description: Enforce locator strategy when writing or reviewing Playwright
  selectors. Use whenever the user asks to add a locator, pick a selector,
  or review existing locators for reliability.
---

# Playwright locator policy

## Priority order
1. `getByRole` (accessible name + role) — always prefer this first.
2. `getByLabel` / `getByPlaceholder` — for form fields without a clear role match.
3. `getByTestId` — only when no accessible attribute exists, using the
   project's `data-testid` convention.
4. Raw CSS/XPath — last resort only; flag it in the response so it can be
   revisited later.

## Process
1. Inspect the snapshot for accessible roles and names before suggesting
   any selector.
2. Never suggest a locator tied to visual position (nth-child, index) —
   these break the moment layout changes.
3. If an existing test uses a fragile selector, suggest a replacement
   rather than leaving it as-is.

Step 3: Skill 2 — Page Object Generator

This Skill turns a page snapshot into a Page Object class matching your team's conventions.

---
name: playwright-page-object-generator
description: Generate or update Playwright Page Object Model classes. Use
  whenever the user asks to scaffold a page object, generate a POM, or
  create a page class for a UI flow.
---

# Playwright page object generator

## Process
1. Navigate to the target page and take a snapshot.
2. Apply `playwright-locator-policy` conventions for every element chosen.
3. Generate one class per page, with:
   - Locators as readonly properties, named after their purpose
     (`emailInput`, not `input1`).
   - One method per user action (`login()`, `submitForm()`), not per
     individual click or fill.
4. Place the file under `pages/<PageName>Page.ts` (or the project's
   existing page object folder, if one exists — check first).
5. Never duplicate an existing Page Object; check for one before creating
   a new file.

Notice this Skill explicitly references the locator-policy Skill's conventions in step 2 — Skills in a pack can lean on each other's standards without repeating the full policy inline.

Step 4: Skill 3 — Flaky Test Debugger

This one handles a completely different trigger: diagnosing, not writing.

---
name: playwright-flaky-test-debugger
description: Diagnose intermittently failing Playwright tests. Use whenever
  the user says a test is flaky, fails intermittently, or passes locally
  but fails in CI.
---

# Playwright flaky test debugger

## Process
1. Ask for (or locate) the failing test's trace file or CI logs first —
   never guess at a root cause without evidence.
2. Check, in order, for the most common causes:
   - Missing `await` before an action or assertion.
   - An assertion that doesn't wait for state (use `expect(locator)`
     auto-waiting patterns, not manual `sleep`/`wait` calls).
   - Test-order dependency (does it fail only when run after another
     specific test?).
   - Environment difference (timing, viewport, or data state between
     local and CI).
3. Report the most likely cause with the specific line of evidence from
   the trace/log that supports it — don't report a guess as a finding.
4. Suggest one fix at a time; don't rewrite the whole test speculatively.

Notice step 3's guardrail — this mirrors the "never fabricate" guidance that's become a best practice for AI-assisted QA work: report evidence, not guesses.

Step 5: Sharing a Common References Folder

If several Skills need the same longer reference (say, your team's full accessibility-name conventions for getByRole), put it once in a shared location instead of duplicating it:

.claude/skills/playwright-shared/references/accessible-naming.md

Then point to it by relative path from any Skill that needs it, rather than pasting the same content into three separate SKILL.md bodies. Per the progressive disclosure model from Part 5, this file still only loads when a step actually calls for it — sharing it doesn't cost you anything extra.

Step 6: Verifying the Whole Pack

Restart your Claude Code session, then test each trigger independently:

Add a locator for the "Remember me" checkbox.
→ should fire playwright-locator-policy

Generate a page object for the checkout page.
→ should fire playwright-page-object-generator

This test passes locally but fails in CI about half the time.
→ should fire playwright-flaky-test-debugger

If two Skills fire for the same request, or neither fires, revisit their description fields — Part 5 covered why this is almost always a description problem, not a bug in the Skill body.

Conclusion

A single Skill solves one repeated conversation. A pack — planned so each Skill has a clear, non-overlapping trigger, and sharing common references — starts to genuinely replace parts of your team's onboarding documentation, all without costing you tokens for the Skills you're not using in a given session. In the next post, we'll add explicit guardrails and human-review gates across this whole pack, so none of these Skills ever ship a fabricated result unchecked.

Have you grouped your own Skills into a pack yet? What's in it? Let me know in the comments!

Progressive Disclosure: Why You Can Install 30 Skills and Pay for Almost Nothing

Aswani Kumar — Wed, 22 Jul 2026 16:30:00 +0000

Part 5 of the "Automating Playwright with Claude Code" series. In Part 4, we built a single playwright-form-tester Skill. This post explains the mechanism that lets you install dozens more like it without bloating every session — and shows how to structure a Skill to actually take advantage of it.

If you've ever wondered "wait, if I install 10 Skills, does Claude load all 10 into context on every single request?" — the answer is no, and the reason why is one of the more elegant parts of how Claude Code Skills are designed: progressive disclosure. Understanding it changes how you write Skills, not just how many you install.

Why Progressive Disclosure Matters

Skill libraries scale for free. You can install 30+ Skills covering your entire test lifecycle without paying a token tax for the ones you're not using right now.
It changes how you should write Skills. Once you know only the frontmatter is always loaded, you write leaner descriptions and push heavy detail into files Claude only opens on demand.
It explains Skill-fires-or-doesn't behavior. If a Skill never triggers, or triggers when it shouldn't, the fix is almost always in this loading model — usually the description.
It's the difference between a Skill and a bloated CLAUDE.md. A CLAUDE.md file is always fully in context. A Skill isn't — and that's the entire point.

Prerequisites
Completed Part 4 of this series, with the playwright-form-tester Skill installed.
Comfortable editing a SKILL.md file and creating subfolders alongside it.

Table of Contents

The Three Layers of a Skill
Step 1: What's Always in Context (Layer 1)
Step 2: What Loads on Match (Layer 2)
Step 3: What Loads Only on Demand (Layer 3)
Step-by-Step: Refactoring Our Form-Tester Skill
Why the Description Field Is the Most Important Line You'll Write
Conclusion

The Three Layers of a Skill

Every installed Skill loads in three stages, only as needed:

Layer	What's in it	When it loads
1. Frontmatter	`name` + `description` (~100 tokens)	Always, for every Skill, at session start
2. Body	The full `SKILL.md` instructions	Only when your request matches the description
3. Files	`references/`, `scripts/`, `assets/`	Only when a step in the body actually calls for that specific file

This is why installing 30 Skills doesn't mean paying for 30 Skills' worth of context on every request — you're only ever paying for Layer 1 across the board, plus Layers 2 and 3 for the one or two Skills that actually apply to what you asked.

Step 1: What's Always in Context (Layer 1)

At the start of every Claude Code session, only this much of each installed Skill gets loaded:

---
name: playwright-form-tester
description: Test HTML forms using Playwright CLI. Use this whenever the user
  asks to test, validate, or verify a form (login, signup, checkout, contact,
  etc.) on a web page, or mentions form submission, validation errors, or
  success messages.
---

That's it — no workflow steps, no code blocks, nothing else.
Multiply this by however many Skills you have installed, and you can see why 30 Skills costs roughly 30 × ~100 tokens, not 30 × (full SKILL.md length).

Step 2: What Loads on Match (Layer 2)

The moment you say something like "test the checkout form", Claude matches it against the description field above, and only then pulls in the full body:

## Process
1. Navigate to the target page with `playwright-cli navigate <url>`.
2. Run `playwright-cli snapshot` to get element references (e.g. `e12`).
3. Fill each field with `playwright-cli fill <ref> "<value>"`.
...

Every other installed Skill that didn't match stays at Layer 1 — its body is never loaded for this request.
This is the point where your Skill starts costing real tokens, but only for the one request where it's relevant.

Step 3: What Loads Only on Demand (Layer 3)

This is the layer most people don't take advantage of, and it's the one that matters most once a Skill grows complex. If a step in your body references a file — a script, a longer reference doc, a template — that file is only opened if that specific step actually runs.

.claude/skills/playwright-form-tester/
├── SKILL.md
├── scripts/
│   └── check-known-selectors.sh
└── references/
    └── negative-test-checklist.md

A script's execution output is what enters context — not its source code. Claude runs check-known-selectors.sh and only sees the result, not the whole script.
references/negative-test-checklist.md only gets read if the workflow step says "consult the negative test checklist" — otherwise it sits on disk, costing nothing.

Step-by-Step: Refactoring Our Form-Tester Skill

Let's apply this to the Skill from Part 4. Right now, our negative-testing guidance is baked directly into the body:

## Notes
- Always test one valid case and at least one invalid case per form.
- If the form has a CAPTCHA or OTP step, stop and ask the user how to
  proceed rather than guessing.

That's short enough to leave inline — but imagine it grew into a 40-line checklist covering SQL injection payloads, Unicode edge cases, and field-length boundaries. At that size, it belongs in a reference file instead:

Create the file:

mkdir -p .claude/skills/playwright-form-tester/references

Move the detailed checklist into references/negative-test-checklist.md.
Point to it from the body instead of inlining it:

## Notes
- Always test one valid case and at least one invalid case per form.
- For a full negative-test checklist (injection payloads, boundary
  values, Unicode edge cases), consult
  `references/negative-test-checklist.md`.

Now that checklist only enters context on the requests where Claude actually needs it — not on every single form test.

Why the Description Field Is the Most Important Line You'll Write

If a Skill isn't firing when you expect it to, the description is almost always the reason — because it's the only thing Claude has to go on before deciding to load the rest.

# Too vague — likely won't fire reliably
description: Helps with testing.

# Specific, with real trigger phrases
description: Test HTML forms using Playwright CLI. Use this whenever the user
  asks to test, validate, or verify a form (login, signup, checkout, contact,
  etc.) on a web page, or mentions form submission, validation errors, or
  success messages.

Include the actual words a tester would type — "test the login form," "validate this signup flow" — not just an abstract summary.
If you have several similar Skills, make sure their descriptions don't overlap so much that Claude can't tell them apart.

Conclusion

Progressive disclosure is why Skills scale where a giant CLAUDE.md file doesn't — you get the benefit of a large, well-organized library while only ever paying for the small slice that's relevant to what you're doing right now. In the next post, we'll use this exact structure to grow our single form-tester Skill into a small pack covering page objects, flaky-test debugging, and locator strategy — much like the framework packs the wider QA-skills ecosystem has started building.

Have you restructured a Skill to use references/ or scripts/ yet? Let me know how it went in the comments!

🤖 AI-Assisted Test Case Generation using Playwright & GPT APIs

Aswani Kumar — Mon, 20 Jul 2026 16:40:00 +0000

Writing and maintaining test cases manually can be time-consuming, especially in fast-paced Agile environments. What if we could automate even test creation using AI?

In this blog, you'll learn how to build an AI-powered assistant that uses OpenAI’s GPT API to automatically generate Playwright test cases based on natural language requirements.

Let's dive into the future of intelligent test automation.

🧠 Why AI for Test Case Generation?

Test creation involves:

Understanding user flows.
Translating them into code.
Selecting appropriate locators.
Handling validations, waits, and edge cases.

AI can assist by:

Parsing user stories or acceptance criteria.
Suggesting Playwright test templates.
Adapting locator strategies.
Speeding up test authoring.

🏗️ Architecture Overview

User Story / Prompt
   |
   v
Prompt Handler → GPT API → Test Code Generator
   |
   v
Playwright Test Output (.ts)

🔌 Sample Use Case

🧾 Input Prompt

"Test login flow where user visits login page, enters valid username and password, clicks Login, and sees dashboard."

🤖 AI-Generated Output

import { test, expect } from '@playwright/test';

test('Login flow', async ({ page }) => {
  await page.goto('https://example.com/login');
  await page.fill('input[name="username"]', 'testuser');
  await page.fill('input[name="password"]', 'securepassword');
  await page.click('button[type="submit"]');
  await expect(page).toHaveURL('https://example.com/dashboard');
});

⚙️ Implementation Steps

1. 🔑 Setup OpenAI

npm install openai dotenv

Create a .env file:

OPENAI_API_KEY=your_openai_key_here

2. 📦 Create Script to Send Prompt

// ai-generator.ts
import { OpenAI } from 'openai';
import * as fs from 'fs';
import * as dotenv from 'dotenv';
dotenv.config();

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

const prompt = `Generate a Playwright test case:
Scenario: User signs up with email and password`;

async function generateTest() {
  const completion = await openai.chat.completions.create({
    messages: [{ role: 'user', content: prompt }],
    model: 'gpt-4',
  });

  const testCode = completion.choices[0].message.content;
  fs.writeFileSync('tests/generated.spec.ts', testCode);
  console.log('✅ Test created: tests/generated.spec.ts');
}

generateTest();

🧪 Output Folder Structure

playwright-ai-generator/
├── tests/
│   └── generated.spec.ts     # AI-created test
├── ai-generator.ts           # GPT-based generator
├── .env
├── package.json
└── playwright.config.ts

🌟 Advanced Enhancements

🧠 Add NLP pre-processing for structured prompts.
📸 Ask GPT to match selectors from screenshots or DOM.
🧾 Include validation and edge case hints in prompt.
🛠️ Combine with self-healing logic for production resilience.

🚨 Caveats & Cautions

Always review AI-generated tests for correctness.
GPT may use outdated syntax unless prompted specifically.
Sensitive data (e.g., credentials) should be mocked or parameterized.

📌 Final Thoughts

AI won't replace test engineers — but it can turbocharge productivity. By combining Playwright’s power with GPT's intelligence, you can:

Rapidly prototype tests.
Build internal QA copilots.
Reduce time spent on repetitive test creation.

🚑 Creating a Self-Healing Automation Framework with Playwright

Aswani Kumar — Sun, 19 Jul 2026 16:36:00 +0000

🧠 What is a Self-Healing Automation Framework?

A self-healing test framework can automatically detect and recover from broken locators or test failures — reducing test flakiness and manual maintenance effort.

Instead of failing immediately when a selector is not found, the framework:

Detects similar or alternate selectors intelligently.
Applies fallback strategies to continue execution.
Logs the issue for review and permanent fix.

🚀 Why Use Playwright for Self-Healing?

Playwright, by design, offers:

Robust locator strategies (e.g., getByRole, getByText, nth, etc.).
Auto-waiting mechanism for stability.
Built-in retries for flaky steps.

This makes it a perfect base for adding self-healing intelligence on top.

🏗️ Architecture of a Self-Healing Playwright Framework

Here’s the high-level design you can follow:

Test Scripts
   |
   v
Custom Locator Wrapper (with Healing Logic)
   |
   v
Locator Engine (Playwright)

Key Components:

Locator Map: A JSON or database file that stores element selectors with possible alternatives.
Smart Wrapper: A utility that tries primary and fallback locators in sequence.
Logging Layer: To log failed attempts, auto-fixes, and suggestions.
AI/ML Module (Optional): Use AI to suggest closest matches if none are found.

🛠️ Code Implementation (Simplified Example)

📁 `locatorMap.ts` (Locator Store)

export const locatorMap = {
  "loginButton": [
    "//button[text()='Login']",
    "//button[contains(@class, 'btn-login')]",
    "button:has-text('Sign In')"
  ],
  "usernameInput": [
    "input[name='username']",
    "#user",
    "//input[@placeholder='Username']"
  ]
};

📁 `healingLocator.ts` (Wrapper Function)

import { Page } from '@playwright/test';
import { locatorMap } from './locatorMap';

export async function findWithHealing(page: Page, key: string) {
  const locators = locatorMap[key];

  for (const locator of locators) {
    try {
      const element = page.locator(locator);
      if (await element.count() > 0) {
        console.log(`[Healed] Found "${key}" using: ${locator}`);
        return element;
      }
    } catch (err) {
      continue;
    }
  }

  throw new Error(`Element "${key}" not found via any fallback`);
}

✅ Usage in Tests

import { test } from '@playwright/test';
import { findWithHealing } from './healingLocator';

test('Login flow with healing selectors', async ({ page }) => {
  await page.goto('https://example.com/login');

  const username = await findWithHealing(page, 'usernameInput');
  await username.fill('aswani');

  const loginBtn = await findWithHealing(page, 'loginButton');
  await loginBtn.click();
});

🧩 Bonus: AI-Based Self-Healing (Optional Layer)

For advanced teams, integrate AI tools like OpenAI or fuzzy string matching to:

Suggest closest matching elements.
Auto-learn updated selectors from DOM diffs.
Store successful fallback attempts for self-learning.

👉 Example prompt to GPT:

“Suggest an alternate selector for a missing button labeled ‘Login’ in a changed DOM structure.”

📊 Benefits of Self-Healing Framework

🔄 Reduced Test Failures due to DOM changes.
💡 Proactive Maintenance with fallback logging.
📉 Lower Maintenance Cost and manual rework.
⏱️ Faster Feedback Loops in CI/CD pipelines.
🤖 Better Stability in Agile Teams that release often.

🚨 When NOT to Use Self-Healing

Self-healing is a safety net, not a permanent bandage.

Avoid using it:

As a replacement for good test design.
When the application is still evolving rapidly.
If false positives can harm critical workflows.

📌 Final Thoughts

A self-healing Playwright framework bridges the gap between rapid development and test stability. While it won’t replace smart test design, it adds resilience to your automation suite.

If your QA team is struggling with flaky tests, try implementing this approach. Start simple, monitor results, and expand with AI if needed.

💬 What’s Next?

✅ Want a GitHub template for this self-healing setup? Drop a comment below.

📩 Have questions about integrating AI with Playwright? Let’s connect!

📁 GitHub Project Structure (Planned)

We will provide a full working Playwright project with:

A main branch: Standard test automation setup.
A self-healing branch: Enhanced with fallback locator support.
Demo tests using a public login form.

🔗 GitHub link coming soon — follow for updates!

Integrating Playwright with Other Tools (Allure, Postman, etc.)

Aswani Kumar — Sat, 18 Jul 2026 16:33:00 +0000

Introduction
1. Integrating Playwright with Allure for Rich Test Reports
- Step-by-Step Integration:
  - 1. Install Allure Playwright Adapter
  - 2. Update Playwright Config
  - 3. Generate and View the Report
  - 4. CI/CD Compatibility
2. Integrating Playwright with Postman for API + UI Testing
- Option A: Use Newman CLI to Run Postman Collections
  - 1. Export Postman Collection and Environment
  - 2. Install Newman
  - 3. Run Collection in Test Script
- Option B: Migrate Postman Tests to Playwright API Testing
3. Integrating Playwright in CI/CD Pipelines
4. Bonus: Integrate with Slack for Notifications
Conclusion

Introduction

Playwright has quickly become a go-to automation tool for modern web testing due to its speed, reliability, and cross-browser support. However, the real power of Playwright comes to life when it is integrated with other tools like Allure for reporting, Postman for API testing, and CI/CD systems for continuous testing.

In this blog post, we’ll walk through practical and effective ways to integrate Playwright with essential testing and reporting tools to enhance your automation workflows.

1. Integrating Playwright with Allure for Rich Test Reports

Allure provides visually appealing and detailed test reports that can be extremely useful for debugging and stakeholder communication.

Step-by-Step Integration:

1. Install Allure Playwright Adapter

npm install -D allure-playwright

2. Update Playwright Config

Add the Allure reporter to your playwright.config.ts or .js file:

import { defineConfig } from '@playwright/test';

export default defineConfig({
  reporter: [ ['allure-playwright'] ]
});

3. Generate and View the Report

After running your tests, generate the report:

npx allure generate ./allure-results --clean -o ./allure-report
npx allure open ./allure-report

4. CI/CD Compatibility

These reports can be stored as artifacts or published as HTML pages in Jenkins, GitLab, or GitHub Actions.

2. Integrating Playwright with Postman for API + UI Testing

Combining API tests from Postman with UI tests in Playwright helps validate both frontend and backend behavior in your application.

Option A: Use Newman CLI to Run Postman Collections

1. Export Postman Collection and Environment

Download your.json files from Postman.

2. Install Newman

npm install -g newman

3. Run Collection in Test Script

import { execSync } from 'child_process';

test.beforeAll(() => {
  execSync('newman run collection.json -e environment.json');
});

Option B: Migrate Postman Tests to Playwright API Testing

Playwright has native support for API testing:

const response = await request.get('https://api.example.com/users');
expect(response.ok()).toBeTruthy();

This eliminates external dependencies and improves execution speed.

3. Integrating Playwright in CI/CD Pipelines

CI/CD integration ensures your Playwright tests run on every commit or deployment.

GitHub Actions

name: Playwright Tests
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Install dependencies
        run: npm ci
      - name: Run Playwright tests
        run: npx playwright test
      - name: Upload Allure report
        run: |
          npx allure generate ./allure-results --clean -o ./allure-report
          npx allure open ./allure-report

GitLab CI

test:
  script:
    - npm ci
    - npx playwright install
    - npx playwright test
    - npx allure generate ./allure-results --clean -o ./allure-report
  artifacts:
    paths:
      - allure-report

4. Bonus: Integrate with Slack for Notifications

Keep your team updated with test status by sending Slack notifications.

Slack Notifier (via webhook):

import fetch from 'node-fetch';

async function sendSlackNotification(status: string) {
  await fetch('https://hooks.slack.com/services/your/webhook/url', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ text: `Playwright Tests: ${status}` })
  });
}

Call sendSlackNotification('✅ Passed') or sendSlackNotification('❌ Failed') based on test results.

Conclusion

Integrating Playwright with Allure, Postman, CI/CD, and Slack supercharges your automation strategy. These integrations allow you to go beyond simple test execution and build a holistic testing ecosystem that ensures rapid feedback, reliability, and enhanced team collaboration.

What integrations have made your Playwright automation more powerful? Share your thoughts and setups in the comments!

What Is SKILL.md? Building a Reusable Playwright Testing Skill for Claude Code

Aswani Kumar — Fri, 17 Jul 2026 16:30:00 +0000

Part 4 (final) of the "Automating Playwright with Claude Code" series. If you're just joining, start with Part 1: Getting Started with Claude Code and Playwright CLI, then Part 2: CLI vs MCP and Part 3: Token-Saving Habits.

Across this series, we've set up Playwright CLI, compared it against MCP, and covered habits for keeping Claude Code sessions token-efficient. One idea kept coming up: pushing repeatable steps into a Skill instead of re-explaining them every session. This final post shows exactly what that means — what SKILL.md is, how to write one, and a full working example you can drop straight into your own project.

Why Skills Matter for Test Automation

Consistency across the team. Once a Skill is committed to your repo, every teammate gets the same testing conventions automatically — no more "well, I usually test forms by..." tribal knowledge.
Context stays lean. A Skill's instructions only load when Claude actually decides they're relevant — unlike a CLAUDE.md file, which is always present in every session.
Token-efficient by design. Skills can bundle scripts and reference files that Claude only reads on demand, keeping routine work cheap even across many sessions.
Portable and shareable. It's an open markdown-based format, so the same Skill folder works across projects and, increasingly, across other agent tools too.
No new syntax to memorize. You don't need to remember a slash command — Claude recognizes when a Skill applies based on your plain-language request.

Prerequisites

Claude Code installed (see Part 1).
Playwright CLI installed, ideally with playwright-cli install --skills already run.
A project you're comfortable committing a .claude/skills/ folder into.

What Is SKILL.md?
Anatomy of a Skill File
Step-by-Step: Building a Playwright Form-Testing Skill
Testing That Your Skill Actually Fires
Where to Put Skills: Project vs Personal
Advantages of Using Skills, Recap
Conclusion

Step 1: What Is SKILL.md?

A Skill in Claude Code is a folder — typically containing a SKILL.md file — that packages instructions Claude loads only when it's relevant to the task at hand, rather than sitting in context for every single session:

.claude/skills/
└── playwright-form-tester/
    └── SKILL.md

Think of it as documentation written specifically for Claude to read at the right moment: not a static reference you consult, but a file Claude actively decides to pull into its own context based on what you ask.

Step 2: Anatomy of a Skill File

Every SKILL.md has two parts:

---
name: skill-name-here
description: "A clear description of what this does and when to use it."
---

# Skill instructions in markdown go here.

YAML frontmatter — name and description. The description is the most important field in the entire file: Claude reads every installed Skill's description at session start to decide which one(s) to load for a given request. Vague descriptions mean the Skill just won't fire.
Markdown body — the actual process, conventions, or knowledge you want followed once the Skill is triggered. Plain instructions, same as you'd give a new team member.

Step 3: Building a Playwright Form-Testing Skill

Let's build a real one — a Skill that teaches Claude your team's standard approach to testing any form using Playwright CLI (the tool from Parts 1–2 of this series).

---
name: playwright-form-tester
description: Test HTML forms using Playwright CLI. Use this whenever the user
  asks to test, validate, or verify a form (login, signup, checkout, contact,
  etc.) on a web page, or mentions form submission, validation errors, or
  success messages.
---

# Playwright form tester

## When to use this
Any time a form needs to be exercised end-to-end: filling fields, submitting,
and checking the result (success message, redirect, or validation error).

## Process
1. Navigate to the target page with `playwright-cli navigate <url>`.
2. Run `playwright-cli snapshot` to get element references (e.g. `e12`).
3. Fill each field with `playwright-cli fill <ref> "<value>"`.
4. For negative test cases, submit with at least one required field empty
   and confirm a validation message appears — check for the presence of
   the specific error text, not just the absence of errors.
5. Submit the form and re-run `playwright-cli snapshot` to confirm the
   expected end state (success message, redirect URL, or error banner).
6. Report pass/fail clearly, quoting the exact text found on the page.

## Notes
- Always test one valid case and at least one invalid case per form.
- If the form has a CAPTCHA or OTP step, stop and ask the user how to
  proceed rather than guessing.

Save this exactly as .claude/skills/playwright-form-tester/SKILL.md in your project.

Step 4: Testing That Your Skill Actually Fires

Skills load at session start, so:

Restart your Claude Code session (or start a new one) after adding the file.
Ask something that matches the description, e.g.:

Test the checkout form on staging.

Claude should recognize the match and follow your defined process — navigating, filling fields, checking both a valid and invalid case, and reporting results in the format you specified.

If it doesn't fire, double-check:

The file is named exactly SKILL.md (case-sensitive on some systems).
The frontmatter is valid YAML (watch for missing colons or bad indentation).
The description field actually contains the trigger words someone would naturally use.

Step 5: Where to Put Skills — Project vs Personal

Location	Scope	Use for
`.claude/skills/` in your repo	Project-only, shared with team via git	Testing conventions specific to this app
`~/.claude/skills/`	Personal, available across all your projects	Your own general-purpose habits (e.g. "always test one negative case")

Commit project-scoped Skills to version control so the whole team benefits the moment they pull the branch — no separate setup instructions needed.

Step 6: Advantages of Using Skills — Recap

Keeps context lean by loading only when relevant, unlike an always-present CLAUDE.md.
Makes testing conventions consistent across an entire team, automatically.
Turns tribal knowledge ("we usually test forms this way") into a versioned, shareable artifact.
Works alongside everything else in this series — pair it with Playwright CLI (Part 1–2) and the token-saving habits (Part 3) for genuinely efficient, repeatable test automation sessions.

Conclusion

That wraps up the series: Playwright CLI setup, how it compares to MCP, the everyday habits that keep sessions efficient, and now Skills — the piece that makes all of it repeatable and shareable across your team. If you build a Skill from this template, I'd love to see how you've adapted it for your own project's forms and flows.

Have you built a Skill of your own yet? Share what it does in the comments — and thanks for following along with the whole series!

How to Save Tokens in Claude Code: Practical Habits for Test Automation Sessions

Aswani Kumar — Thu, 16 Jul 2026 16:30:00 +0000

Part 3 of the "Automating Playwright with Claude Code" series. Catch up on Part 1: Getting Started with Claude Code and Playwright CLI and Part 2: Playwright CLI vs MCP if you missed them.

In Part 2, we saw nearly a 90,000-token gap between Playwright MCP and Playwright CLI on the exact same test. But tool choice is only one lever. Once you're running longer test-writing sessions with Claude Code — building out a whole regression suite, not just one test — a handful of everyday habits make just as much difference to your token bill and your context window. This post covers the ones that consistently pay off.

Why Token Usage Matters for Test Automation

Long sessions hit context limits. A regression suite with 15+ forms to test can genuinely exhaust a session if every step re-sends unnecessary state.
Cost scales with usage. If you're on API pricing rather than a flat subscription, token-heavy sessions translate directly into a bigger bill.
Wasted tokens mean wasted time. Once a session gets bloated, responses slow down and Claude has to work harder to find the relevant thread in a cluttered context.
Good habits compound. None of these tips are dramatic on their own, but stacked together across a full day of test-writing, they add up to meaningfully longer, cheaper sessions.

Prerequisites

Claude Code installed and set up (see Part 1 if you need the Playwright CLI setup too).
Comfortable running slash commands (/clear, /compact) inside a Claude Code session.

Scope Every Task Tightly
Use /clear Between Unrelated Tasks
Use /compact Instead of Letting Context Grow Unchecked
Prefer Headless Mode by Default
Push Repeatable Work into Scripts and Skills
Pin Tool Versions in CI
Conclusion

Step 1: Scope Every Task Tightly

The single biggest lever is also the simplest: be specific about what Claude should touch.

❌ Explore the whole app and test everything you find.
✅ Test the login form at /login: valid credentials, empty password, and wrong password cases.

A broad, open-ended instruction invites Claude to navigate multiple pages, take extra snapshots, and explore paths you didn't actually need tested — all of which costs tokens.
A scoped instruction gets a scoped result: Claude touches exactly the pages and elements relevant to that form.
If you do want broader exploration, do it as its own deliberate step ("first, list every form on this page"), not bundled into the same request as the actual testing.

Step 2: Use `/clear` Between Unrelated Tasks

/clear

Run this whenever you're switching from one unrelated task to another — say, finishing the login form tests and starting on the checkout flow.
Without it, Claude's context keeps accumulating snapshot history, tool outputs, and conversation turns from the previous task, none of which is relevant to the next one.
Rule of thumb: if the next thing you're about to ask has nothing to do with what you just finished, clear first.

Step 3: Use `/compact` Instead of Letting Context Grow Unchecked

/compact

For tasks that genuinely need the earlier context (you're still working through the same feature, just many steps in), /compact summarizes the conversation so far instead of wiping it — you keep continuity without carrying every raw tool output forward.
Use this proactively in long sessions rather than waiting until you hit a context warning.

Step 4: Prefer Headless Mode by Default

Headed (visible) browser sessions are useful when you're actively debugging and want to watch what's happening, but they tend to invite more exploratory screenshotting and back-and-forth.
For routine test writing where you already know the flow, default to headless and only switch to headed mode when something's actually failing and you need to see why.

Step 5: Push Repeatable Work into Scripts and Skills

If you find yourself explaining the same steps to Claude across multiple sessions — say, your team's standard login flow before every test — that's a sign it belongs in a script or a Skill, not in a fresh prompt every time.

.claude/skills/playwright-form-tester/SKILL.md

A script's output is what costs tokens, not the logic that produced it — so deterministic steps run far cheaper as a script than as freshly reasoned-through instructions.
A Skill only loads into context when it's actually relevant, so you're not paying for it on unrelated tasks either.
We'll build a full example Skill for this exact use case in Part 4 of this series.

Step 6: Pin Tool Versions in CI

npm install -g @playwright/cli@1.2.0

Using @latest in CI means an upstream update can silently change behavior mid-pipeline, causing retries — and retries cost tokens twice over.
Pin explicit versions for both Playwright CLI and your browser binaries in CI configs, and upgrade deliberately rather than automatically.

Conclusion

None of these habits are complicated, but together they're the difference between a Claude Code session that runs out of steam halfway through a regression suite and one that comfortably gets through your whole day's testing. In the next and final post of this series, we'll put several of these ideas together and build a real, reusable Skill for Playwright form testing — the SKILL.md file that ties this whole series together.

What's your current token-saving habit in Claude Code — anything I missed? Let me know in the comments!

Playwright CLI vs Playwright MCP: Which Should You Use with Claude Code?

Aswani Kumar — Wed, 15 Jul 2026 16:30:00 +0000

If you searched "Claude Code + Playwright," the first thing you probably found was the Playwright MCP server — it's the original, most widely documented integration. But there's a newer option, Playwright CLI, built specifically for agents like Claude Code that already have shell access. They solve the same problem in very different ways, and picking the wrong one for your situation can cost you real time and real tokens.

This post breaks down exactly how they differ, with real numbers, so you can pick the right tool for your project.

⚡ Update — 24th July 2026 (read this first)

Since I first published this, the numbers below shifted, and I want the post to stay honest — so here's the correction up front.

I re-ran the benchmark on the current Playwright MCP server (@playwright/mcp@0.0.78) against @playwright/cli@0.1.17. The ~4× token gap has essentially closed. Modern Playwright MCP no longer streams the full page state into the conversation on every call — like the CLI, it now writes the snapshot to disk and returns a short file path. On an identical login → dashboard flow the two tools landed within ~1% of each other, not 114k vs 27k.

The original numbers further down were a real measurement of older MCP behavior, so I've kept them for context and clearly marked them as historical. The full re-benchmark is in Step 4. Bottom line for 2026: choose between MCP and CLI based on filesystem access, not on token cost.

Why This Comparison Matters

Token cost adds up fast in long test-writing sessions — the wrong tool can burn your budget on a single flow.
Not every environment supports both — sandboxed clients like Claude Desktop can't run CLI commands, so MCP is your only option there.
CI pipelines behave differently than interactive chat sessions, and one of these tools fits CI workflows much better than the other.
Knowing both means you're not locked into a single approach across your whole test suite.

Prerequisites

Completed Part 1 of this series, with Playwright CLI installed and working.
Claude Code installed and authenticated.
(Optional, for comparison) Playwright MCP installed: claude mcp add playwright npx @playwright/mcp@latest

What Is Playwright MCP?
What Is Playwright CLI?
Step-by-Step: The Same Test, Both Ways
Token Usage Comparison
Decision Table: Which One Should You Use?
Conclusion

Step 1: What Is Playwright MCP?

Playwright MCP is a Model Context Protocol server that gives Claude tools like browser_navigate, browser_click, and browser_snapshot. It's the plug-and-play option:

claude mcp add playwright npx @playwright/mcp@latest

Once registered, Claude calls these tools directly inside the conversation — no shell access required.
In older versions, the full accessibility tree (and often screenshots) was returned as part of the tool result — injected straight into the model's context on every call. That's no longer the case: current MCP writes the snapshot to disk and returns a file path instead (see the Step 4 update).
This makes MCP the right choice for sandboxed environments — Claude Desktop, custom chat UIs, or any client without terminal access.

Step 2: What Is Playwright CLI?

Playwright CLI is a newer, standalone command-line tool built for coding agents that already have filesystem and shell access:

npm install -g @playwright/cli@latest
npx playwright install chromium

Instead of streaming page state into the conversation, it writes a compact YAML snapshot to disk.
Claude reads only the specific lines it needs from that file (via the CLI's find command), instead of the whole document.
Because it's just shell commands, it works anywhere Claude Code already runs — locally or in CI.

Step 3: The Same Test, Both Ways

Here's identical logic — navigate, fill a field, submit, verify — expressed through each tool.

With MCP (called as conversational tool-use, no shell commands visible):

Claude calls: browser_navigate(url="https://demo.playwright.dev/todomvc")
Claude calls: browser_snapshot()
Claude calls: browser_type(target="e12", text="Write blog post")
Claude calls: browser_press_key(key="Enter")
Claude calls: browser_snapshot()

With CLI (run as literal shell commands):

playwright-cli open https://demo.playwright.dev/todomvc
playwright-cli snapshot
playwright-cli fill e12 "Write blog post" --submit
playwright-cli snapshot

Same steps, same underlying accessibility-tree referencing system (e12 means the same thing in both — they share the same playwright-core tooling).
The difference isn't what happens — it's where the resulting state lives and how much of it re-enters the model's context.

Step 4: Token Usage Comparison

Updated 24th July 2026 — read this before the original numbers below.

When I first wrote this, Playwright MCP returned the full accessibility tree inline on every tool call, which is where the eye-watering numbers came from. The current MCP server (@playwright/mcp@0.0.78) doesn't do that anymore — it writes the snapshot to disk and returns a file path, exactly like the CLI. I re-benchmarked both on an identical login → dashboard flow (navigate → snapshot → type username → type password → click → snapshot → screenshot):

Tool (current versions) Chars returned into context, same 7-step flow

Playwright MCP 0.0.78 ~3,580

Playwright CLI 0.1.17 ~3,620

Within ~1% of each other, step for step. The clearest proof is a single navigate on a heavy 300-row page: both tools return only a ~230-character path to an on-disk snapshot, even though the full accessibility tree for that page is ~97,000 characters. That heavy tree only enters context if the agent explicitly asks for a full snapshot — and the token-efficient path (a scoped find) pulls ~450 characters in either tool.

The one real difference left is fixed startup overhead: MCP keeps its ~24 tool definitions (~18k characters) in the session's tool list the whole time, while the CLI reads a skill file once (~12.6k characters) and is otherwise just shell commands. So the honest 2026 takeaway is: pick based on whether your agent has filesystem access, not on per-action token cost.

Tool (current versions)	Chars returned into context, same 7-step flow
Playwright MCP `0.0.78`	~3,580
Playwright CLI `0.1.17`	~3,620

Original benchmark (historical — reflects older MCP versions):

Independent benchmarking on a typical multi-step browser task found:

Tool	Approx. Tokens for a Multi-Step UI Test
Playwright MCP	~114,000
Playwright CLI	~26,000–27,000

That was close to a 90,000-token difference for the same task — because at the time MCP re-sent full page state into the conversation on every single tool call, while CLI kept that state on disk and Claude only pulled in the specific slice it needed. That exact mechanism is what changed: modern MCP now keeps state on disk too, which is why the re-benchmark above comes out roughly even.

For a one-off exploratory test, this difference barely mattered even then. Today, for a full regression suite the two are comparable per action — so the deciding factor is your environment and workflow, covered in the decision table below.

Step 5: Decision Table — Which One Should You Use?

Use MCP when...	Use CLI when...
Your agent is sandboxed with no shell access (Claude Desktop, custom chat UI)	Your agent has shell/filesystem access (Claude Code, Copilot, Cursor)
You need quick, exploratory, one-off browser tasks	You want the lightest fixed footprint and shell-native automation you can also run in CI
You want conversational back-and-forth over page structure	You want repeatable automation that composes with other shell/file steps

In practice, most teams don't pick just one — they keep CLI as the default for test writing and CI, and leave MCP configured as a fallback for quick interactive debugging or sandboxed clients where a shell isn't available.

Conclusion

Playwright MCP and Playwright CLI aren't competitors so much as tools for different moments — MCP for sandboxed, conversational exploration, and CLI for shell-native, CI-friendly automation. The big token gap that used to separate them has largely closed now that MCP keeps page state on disk, so in 2026 the choice comes down to environment and workflow rather than raw token cost. Now that you know the tradeoffs, in the next post we'll go further into general token-saving habits for Claude Code sessions — beyond just this one tool choice.

Which one are you already using — MCP, CLI, or both? Let me know in the comments!

Getting Started with Claude Code and Playwright CLI: A Step-by-Step Guide

Aswani Kumar — Tue, 14 Jul 2026 16:30:00 +0000

AI coding agents like Claude Code are changing how SDETs approach browser automation. Instead of writing every locator and assertion by hand, you can describe a test scenario in plain English and let Claude drive the browser for you. In this guide, we'll set up Claude Code with Playwright CLI — Microsoft's token-efficient, agent-friendly companion to Playwright — and run our first automated browser test.

By the end of this guide, you'll have Claude Code testing a real form on a real page, using nothing but shell commands under the hood.

Why Use Claude Code with Playwright CLI?

Natural language test execution: Describe what to test instead of writing every selector manually.
Token efficiency: Playwright CLI writes browser state to disk instead of streaming it into the AI's context on every action, which keeps sessions cheaper and longer-running.
No new protocol to learn: It runs as plain shell commands, the same way Claude Code already runs git or npm.
Skill-based workflows: Pairs directly with Claude Code Skills (more on that in a later post), so your team can reuse the exact same testing conventions across projects.
CI-ready: The same commands that work interactively in your terminal can run inside a pipeline.

Prerequisites
Step 1: Install Node.js and Verify Your Environment
Step 2: Install Playwright CLI
Step 3: Install a Browser Engine
Step 4: Install the Playwright Skill for Claude Code
Step 5: Run Your First Automated Test
Troubleshooting
Conclusion

Prerequisites

Before you begin, make sure you have:

Node.js 18 or newer installed (Playwright CLI requires it; older versions can throw obscure errors).
Claude Code installed and authenticated. If you don't have it yet: npm install -g @anthropic-ai/claude-code
A terminal you're comfortable working in (iTerm2, Windows Terminal, or any modern shell).
A local web app or public URL you want to test against (we'll use a demo TodoMVC app for this walkthrough).

Step 1: Verify Your Environment

Start by confirming your Node.js version:

node --version

You need v18+. If you're on an older version, upgrade before continuing — Playwright CLI depends on newer JavaScript runtime features.
Also confirm Claude Code is installed and working:

claude --version

If that command isn't recognized, install it first with npm install -g @anthropic-ai/claude-code, then re-run the check.

Step 2: Install Playwright CLI

Install the CLI globally so it's available from any project:

npm install -g @playwright/cli@latest

The @latest tag pulls the newest release. For CI pipelines, consider pinning to a specific version (e.g. @playwright/cli@1.2.0) so your pipeline doesn't break when a new version ships mid-run.
Confirm the install worked:

playwright-cli --help

You should see a list of available commands (navigate, snapshot, click, fill, screenshot, and more).

Step 3: Install a Browser Engine

Playwright CLI needs at least one browser binary to control:

npx playwright install chromium

You can swap chromium for firefox or webkit depending on which engine you want to test against.
This step downloads roughly 100–300 MB per browser, so make sure you have disk space and a stable connection.

Step 4: Install the Playwright Skill for Claude Code

This is the step most setup guides skip — and it's the one that actually makes Claude reliable with the CLI's exact command syntax instead of guessing:

playwright-cli install --skills

This drops a SKILL.md file into your Claude Code configuration that documents every available CLI command.
Without this step, Claude may occasionally hallucinate flags or fall back to running raw Playwright test scripts instead of using the CLI directly.
You only need to run this once per machine (or once per project, if you're using project-scoped skills).

Step 5: Run Your First Automated Test

Open a Claude Code session in your project directory:

claude

Now, in plain language, describe the test:

Use the Playwright CLI to open https://demo.playwright.dev/todomvc,
add a todo item called "Write blog post", and confirm it appears in the list.

Behind the scenes, Claude Code will run a sequence like this:

playwright-cli navigate https://demo.playwright.dev/todomvc
playwright-cli snapshot                        # saves page state to disk as YAML
playwright-cli fill e12 "Write blog post"
playwright-cli press e12 Enter
playwright-cli snapshot                        # verify the todo now appears

snapshot captures a compact accessibility-tree representation of the page and saves it to disk — Claude only reads the parts it needs, rather than the whole page being pushed into the conversation.
The e12 reference comes directly from that snapshot; Playwright CLI and Playwright MCP both use this referencing system, so if you've used MCP before, the mental model will feel familiar.
Claude will report back with a plain-language summary: whether the todo was added successfully, and what it saw in the final page state.

Troubleshooting

"Command not found: playwright-cli" — the global install didn't add it to your PATH. Try npx playwright-cli --help instead, or re-run the global install.
Claude falls back to raw Playwright scripts instead of the CLI — confirm the Skill installed correctly with playwright-cli install --skills, and restart your Claude Code session (Skills load at session start).
Browser fails to launch — re-run npx playwright install chromium and check you have enough disk space.
Tests behave differently on your CI server — pin your Playwright CLI and browser versions explicitly rather than relying on @latest.

Conclusion

You now have Claude Code and Playwright CLI working together, with Claude driving a real browser through natural language instructions instead of you hand-writing every step. This is just the foundation — in the next post, we'll compare Playwright CLI against Playwright MCP in detail, including real token benchmarks, so you know exactly when to reach for each one.

Have you tried Claude Code for browser testing yet? Drop your experience — or questions — in the comments below!

The Role of an SDET in Agile and DevOps Teams

Aswani Kumar — Fri, 14 Feb 2025 02:42:13 +0000

Introduction
Who is an SDET?
The Role of an SDET in Agile Teams
The Role of an SDET in DevOps Teams
Key Skills for an SDET in Agile and DevOps Teams
Conclusion

Introduction

In today’s fast-paced software development landscape, the role of a Software Development Engineer in Test (SDET) is more crucial than ever. Agile and DevOps methodologies have revolutionized the way software is built and delivered, emphasizing speed, collaboration, and continuous feedback. SDETs play a vital role in ensuring software quality while keeping up with these rapid changes.

In this blog post, we will explore the responsibilities of an SDET in Agile and DevOps teams, the skills required, and how they contribute to a high-quality, efficient development lifecycle.

Who is an SDET?

An SDET is a hybrid role that blends software development and testing expertise. Unlike traditional testers, SDETs write code to create automated test frameworks, develop tools, and ensure that the software meets quality standards from the start.

The Role of an SDET in Agile Teams

Agile methodologies focus on iterative development, quick releases, and constant collaboration. An SDET contributes to Agile teams in the following ways:

Test Automation Development
- Builds robust automation frameworks for functional, API, and performance testing.
- Ensures tests are integrated into CI/CD pipelines for continuous feedback.
Shift-Left Testing
- Encourages testing early in the development cycle.
- Works closely with developers to write unit and integration tests.
Collaboration with Cross-Functional Teams
- Participates in sprint planning, daily standups, and retrospective meetings.
- Works alongside developers, product managers, and DevOps engineers to define quality standards.
Test Strategy and Planning
- Designs comprehensive test strategies that align with Agile principles.
- Prioritizes test cases based on risk and business impact.

The Role of an SDET in DevOps Teams

DevOps aims to unify development and operations to streamline software delivery. SDETs contribute to DevOps teams by:

CI/CD Pipeline Integration
- Ensuring automated tests are part of the deployment process.
- Writing scripts for test execution in different environments.
Infrastructure as Code (IaC) Testing
- Validating infrastructure configurations and deployments.
- Using tools like Terraform and Ansible to automate test environments.
Performance and Security Testing
- Integrating performance tests to monitor application stability.
- Implementing security testing frameworks to detect vulnerabilities early.
Monitoring and Observability
- Setting up test monitoring dashboards.
- Analyzing logs and metrics to identify flaky tests and failures.

Key Skills for an SDET in Agile and DevOps Teams

To excel as an SDET, you need a combination of technical and soft skills:

Programming Skills: Proficiency in languages like Java, Python, or JavaScript.
Automation Expertise: Hands-on experience with tools like Playwright, Selenium, and Cypress.
CI/CD Knowledge: Familiarity with Jenkins, GitLab CI/CD, or GitHub Actions.
Cloud and Containerization: Experience with AWS, Docker, and Kubernetes.
Collaboration and Communication: Ability to work seamlessly with developers, testers, and DevOps engineers.

Conclusion

SDETs are an integral part of Agile and DevOps teams, bridging the gap between development and quality assurance. By focusing on automation, early testing, and continuous improvement, they ensure that software is delivered quickly and reliably.

What challenges have you faced as an SDET in Agile or DevOps teams? Share your thoughts in the comments below!

DEV Community: Aswani Kumar

From Bug Found to Bug Filed: A Bug-Reporter Skill for Claude Code

Why a Bug-Reporter Skill Is Worth Building

Prerequisites

Table of Contents

Step 1: What a Good Bug Report Actually Needs

Step 2: Building the Bug-Reporter Skill

Step 3: Applying Guardrails from Part 7

Step 4: From a Flaky Test Finding to a Filed Issue

Step 5: Customizing the Template for Your Tracker

Conclusion

Adding Guardrails to Your Skills: Human-Review Gates and "Never Fabricate" Rules

Why Guardrails Matter for Test Automation Skills

Prerequisites

Table of Contents

Three Failure Modes Worth Guarding Against

Step 1: The "Never Fabricate" Rule

Step 2: Evidence-Required Reporting

Step 3: Human-Review Gates for Destructive or Ambiguous Actions

Step-by-Step: Retrofitting Guardrails into the Part 6 Pack

Testing That a Guardrail Actually Holds

Conclusion

Beyond the Form Tester: Building a Skill Pack for Your Whole Playwright Suite

Why Build a Skill Pack Instead of One Big Skill

Prerequisites

Table of Contents

Step 1: Planning the Pack

Step 2: Skill 1 — Locator Policy

Step 3: Skill 2 — Page Object Generator

Step 4: Skill 3 — Flaky Test Debugger

Step 5: Sharing a Common References Folder

Step 6: Verifying the Whole Pack

Conclusion

Progressive Disclosure: Why You Can Install 30 Skills and Pay for Almost Nothing

Why Progressive Disclosure Matters

Prerequisites

Table of Contents

The Three Layers of a Skill

Step 1: What's Always in Context (Layer 1)

Step 2: What Loads on Match (Layer 2)

Step 3: What Loads Only on Demand (Layer 3)

Step-by-Step: Refactoring Our Form-Tester Skill

Why the Description Field Is the Most Important Line You'll Write

Conclusion

🤖 AI-Assisted Test Case Generation using Playwright & GPT APIs

🧠 Why AI for Test Case Generation?

🏗️ Architecture Overview

🔌 Sample Use Case

🧾 Input Prompt

🤖 AI-Generated Output

⚙️ Implementation Steps

1. 🔑 Setup OpenAI

2. 📦 Create Script to Send Prompt

🧪 Output Folder Structure

🌟 Advanced Enhancements

🚨 Caveats & Cautions

📌 Final Thoughts

🚑 Creating a Self-Healing Automation Framework with Playwright

🧠 What is a Self-Healing Automation Framework?

🚀 Why Use Playwright for Self-Healing?

🏗️ Architecture of a Self-Healing Playwright Framework

Key Components:

🛠️ Code Implementation (Simplified Example)

📁 locatorMap.ts (Locator Store)

📁 healingLocator.ts (Wrapper Function)

✅ Usage in Tests

🧩 Bonus: AI-Based Self-Healing (Optional Layer)

📊 Benefits of Self-Healing Framework

🚨 When NOT to Use Self-Healing

📌 Final Thoughts

💬 What’s Next?

Integrating Playwright with Other Tools (Allure, Postman, etc.)

Table of Contents

Introduction

1. Integrating Playwright with Allure for Rich Test Reports

Step-by-Step Integration:

1. Install Allure Playwright Adapter

2. Update Playwright Config

3. Generate and View the Report

4. CI/CD Compatibility

📁 `locatorMap.ts` (Locator Store)

📁 `healingLocator.ts` (Wrapper Function)

Step 2: Use `/clear` Between Unrelated Tasks

Step 3: Use `/compact` Instead of Letting Context Grow Unchecked