DEV Community: Iryna Katmaieva

Playwright Agents: Or Once Again About AI Taking Away Our Job

Iryna Katmaieva — Wed, 15 Oct 2025 08:21:35 +0000

Recently, Playwright introduced Playwright Agents.

In short, it’s an AI-driven companion integrated into VS Code (and also supported in Claude Code and OpenCode) that can plan, generate, and heal your Playwright tests — right inside your IDE.

To get started, all you need is the latest VS Code and an updated Playwright version.

Then just run:

npx playwright init-agents --loop=vscode

After setup, you’ll see three agents available in your Copilot Chat dropdown:

Planner – generates a test plan with ideas and user flows
Generator – creates the actual spec files
Healer – fixes and stabilizes flaky tests

🧭 Step 1 — Planner: “Let’s brainstorm the test plan”

When I first tried the Planner, I typed something simple like:

Explore and generate a test plan for …

It generated a detailed markdown test plan covering scenarios and user flows. Perfect input for the next step — Generator.

⚙️ Step 2 — Generator: “Now implement the tests”

Next, I switched to the Generator agent and asked:

Implement test for 1. Landing Page Loads Successfully (Happy Path)

The agent created a new Playwright spec file and walked me through every step of its reasoning.

It even asked permission before running commands in the terminal — which felt like the AI politely asking, “Mind if I test this?”

And here’s where seniority comes into play:

If you’re a junior, the AI feels like a mentor.

If you’re senior, you realize how much control you still need to maintain.

Your understanding directly defines the quality of its output.

During my tests, I decided to let the AI guide me — pretending I was new to Playwright.

My goal: get a working spec that tests the Netflix landing page.

🚦 Step 3 — First run: “And... it failed”

Of course, it failed.

So I switched to the Healer agent and said:

Run and fix it.

That’s when the magic (and chaos) began.

The Healer started debugging, trying different fixes, and even went as far as creating an HTML mock fixture — just to make the test pass.

At that moment, I had to stop the “AI freedom experiment.”

It was fascinating… but painful to watch.

The model clearly lacked real context, over-engineering solutions like a junior dev who just discovered cy.intercept() for the first time.

🎯 What I Learned

After watching Debbie’s demo on YouTube, it became clear:

Playwright Agents shine when used inside an existing Test Automation System — once your repo already has meaningful specs, helpers, and Page Objects. Then the agents can work contextually, not blindly.

So no, the robots aren’t taking our jobs.

Playwright Agent isn’t your competitor — it’s another tool in your toolbox.

Used right, it can offload the boring parts of your day — setup, boilerplate, debugging — letting you focus on what actually matters: designing smarter tests and building better systems.

But comparing this workflow to what we had 2–3 years ago? It’s night and day.

Instead of searching StackOverflow, we ask LLMs.
Instead of Google, we use Perplexity or NotebookLM.
Instead of writing manual docs, we now discuss instructions with our agent.

💬 So What About You?

Have you tried Playwright Agents yet? How has your workflow evolved over the last two years?

🎥 Related Resource

Watch Debbie’s Playwright Agents Demo on YouTube

The Real-World Test Automation Pyramid Explained

Iryna Katmaieva — Wed, 18 Jun 2025 20:01:58 +0000

You’ve probably seen the test automation pyramid. But in real projects with microservices, flaky UIs, queues, and unpredictable data — things get complicated. So I decided to break it all down… using cats 🐱

The more tests on the bottom, the better balanced your pyramid. And yes, cats help explain it better.

Why the Pyramid Still Matters

Here’s what a test pyramid helps with:

Early Bug Detection
Unit tests catch issues before they snowball. Think of it like catching your cat before they knock over your coffee.

Efficiency
The lower the test layer, the faster the feedback. Unit and component tests run fast and often.

Cost-Effectiveness
E2E tests take more time, infrastructure, and patience. Too many = flakiness and frustration.

Fast Feedback Loops
In CI/CD, speed matters. If you know something broke in <30 seconds, you can fix it before stand-up.

Better Coverage
Each level covers a different dimension: isolated logic, contracts, integration, and user journeys.

Unit Tests (Fast & Isolated)

These are usually developer-owned. Just remember:

Test a single function/class in isolation
Use mocks/stubs to avoid touching real systems
Run in milliseconds

Component Tests (Realistic + Fast)

These are underrated. You test a real UI or service component with mocked APIs or state.

Tools:

Cypress Component Testing
Playwright Component Mode
Unit Testing Frameworks: Jest + React, etc.
for Angular, there’s the built-in @angular/core/testing

🎥 In my YouTube demo, I showed how to test component's logic using Cypress.

Contract vs Integration Tests

Let’s clear this up:

Feature	Contract Tests	API Integration Tests
Purpose	Validate interface expectations	Test real service interaction
Speed	⚡ Fast	🐢 Slower
Scope	Just the interface	Full system
Tools	Pact, PactFlow	REST-assured, Postman, etc.

🧩 Contract tests are perfect for microservices and teams working in parallel. Bonus: they help avoid breaking changes!

Integration Tests

This is where we connect our service to real systems:

🔗 DB integration (e.g., PostgreSQL, MySQL)
📬 Queue integration (e.g., RabbitMQ, Kafka)
🌐 Third-party APIs or integration of our APIs between each other.

Mocked Backend UI Tests

Sometimes, you just want to test the frontend logic without backend surprises.

That’s where mocked backend UI tests shine:

You don’t control test data
Backend is flaky/unavailable
Focus is on UI rendering and logic

Tools:

Cypress
Playwright
MSW (Mock Service Worker)

End-to-End Tests (API & UI)

You still need a few critical path E2E tests.

Think of them like the top cat on the pyramid — majestic, expensive, and best used sparingly.

✅ Use for flows like:

User signup/login
Purchase flow
Dashboard summary

📉 Keep E2E under 5% of your total test suite to avoid flakiness and high maintenance.

How to Apply the Pyramid

Push tests downward
Use contract tests instead of full system setups
Mock what you don’t control
Only E2E the essential stuff
Think of it as cat logic: the less chaos, the better the nap. 😼

Summary

The test pyramid is still relevant — especially if you structure it like this:

Unit: tiny, fast, and many
Component: realistic but still isolated
Contract: scalable and safe
Integration: real DBs, queues, APIs
E2E: only where it counts

🐾 If your test suite feels like chasing cats, it's time to rebalance your pyramid.

🎬 Watch the Full Breakdown

Let me know how your team handles test layering — or if you've tried explaining test automation with cats too 😄

How I prepared for the ISTQB Advanced Test Automation Engineer Exam: Tips and Tricks

Iryna Katmaieva — Mon, 10 Mar 2025 15:56:15 +0000

The ISTQB Test Automation Engineer certification is a valuable credential for anyone looking to validate their skills in test automation. Below is a detailed guide incorporating my personal experience on effective strategies and practical tips to help you pass the exam efficiently.

This exam was updated in 2024, and you can find the details on the official website: ISTQB Official Site.

Understand the Exam Structure
Focus on High-Value Chapters
Use Any Online Study Materials
Practice with AI-Generated Questions
Practical Experience is a Must
Understand Your ‘Why’
Manage Your Time Effectively
Final Thoughts

Understand the Exam Structure

Before diving into preparation, it's crucial to understand the structure of the exam. You can find ISTQB Exam Structure Tables v1.11 for reference.

The ISTQB Test Automation Engineer exam consists of 40 multiple-choice questions with a maximum of 66 points.
You need to achieve a passing score of 65% or higher (i.e., 43 points).
Each question has a different weight, with some being worth more points than others.
Some chapters contribute more heavily to the final score, so prioritizing them can make a difference.

Check the official ISTQB syllabus to know which topics are covered and their weightage in the exam.

Focus on High-Value Chapters

Not all topics in the syllabus carry the same weight. Instead of spending equal time on each section, focus on the most important and high-scoring chapters.

Identify the chapters that contribute the most to the final score: The syllabus consists of 8 chapters, but Chapters 2, 3, and 8 bring you the greatest number of points (see the official exam structure table).
Prioritise understanding key concepts, methodologies, and practical applications.
Allocate study time accordingly, ensuring strong comprehension in areas with heavier weight.

Use Any Online Study Materials

You don't have to rely solely on paid resources—there's a wealth of free content available online:

YouTube Channels: Many test automation professionals share detailed explanations, mock exams, and breakdowns of ISTQB concepts.
Developer Forums: Websites like Stack Overflow, Reddit, and QA-specific forums often have valuable discussions on ISTQB-related topics.
Blogs & Articles: Many experienced automation engineers document their learning experiences and provide helpful exam tips.

These resources can supplement official materials and help you understand complex topics in an easy-to-digest format.

Practice with AI-Generated Questions

Practicing with exam sample questions from the official website is one of the best ways to prepare. Additionally, you can use AI tools to generate additional exam practice questions based on the syllabus.

Benefits include:

Simulating the exam environment with timed tests.
Identifying weak areas by reviewing incorrect answers.
Reinforcing learning through repeated exposure to key concepts.
Double-checking AI responses with the syllabus to eliminate AI-generated hallucinations.

Practical Experience is a Must

Since the exam is heavily focused on test automation practical use cases, having hands-on experience is a must. You’ll need to:

Work on real automation projects (if you haven’t already).
Build and run automated test suites using tools like Selenium, Cypress, Playwright, or any other frameworks.
Implement CI/CD integration for test automation (Jenkins, Azure, GitLab, etc).
Understand test frameworks, scripts, and debugging automation failures.

Understand Your ‘Why’

Preparing for the ISTQB Test Automation Engineer certification requires consistent effort. Define clear goals for why you want this certificate.

Whether it's for career advancement, personal growth, or a specific job requirement, keeping your motivation in mind will help you stay focused and driven throughout your preparation.

Reward yourself for progress (small milestones matter!).
Remind yourself of the career benefits of passing this exam (e.g., salary growth, better job opportunities).

Manage Your Time Effectively

Balancing exam preparation with work and personal projects can be challenging. Here’s how to optimize your study time:

Use an AI tool to generate a personalized study schedule for the exam, ensuring efficient topic coverage.
Study in the morning and focus on course development in the evening, as this structure helps maintain energy and productivity.
Use the Pomodoro technique (study in 25-50 minute intervals with short breaks).
Limit distractions (turn off social media, find a quiet study environment).
Avoid overloading yourself on Fridays (if low on energy, make up for it on weekends).

Final Thoughts

Passing the ISTQB Test Automation Engineer exam requires a well-structured approach, practice, and motivation. Using AI tools can definitely speed up your exam preparation.

It took me 2 weeks after work on a full-time job, so it is possible, and you can do it too!

Feel free to use my playlist of AI-generated podcasts on each chapter of the ISTQB-TAE Syllabus.

Happy studying, and best of luck with your exam! 🚀