DEV Community: AutomationDataCamp

From QA tester to Lead QA: what changes, and what US data can't tell you

AutomationDataCamp — Thu, 22 Oct 2026 13:00:00 +0000

After a few years writing and automating tests, many testers wonder whether to keep going deeper on the tooling or to move into a role that sets test strategy, designs the test architecture and runs the team's approach to quality. This article explains what changes, and what US labor data can and cannot tell you before you step up.

TL;DR

The role change: a lead decides what to test, how to organise it and how to measure it; an architect designs the technical structure that keeps it sustainable.
US data: the Bureau of Labor Statistics projects 6% employment growth for software QA analysts and testers from 2025 to 2035, against 10% for software developers.
The limit: the BLS does not track QA leads or test architects separately, so no official figure exists for those roles.

Tester, lead, architect: three different jobs

Job titles vary from one company to the next, and what follows is my reading of the field, not a standard. A tester designs and runs tests, often with an automation tool, on the scope they are given. A QA lead answers a different question: for this product, with this deadline and this budget, what should we test, at which level, and who does it? A test architect looks after structure: how test code is organised, where data comes from, how tests run in the CI pipeline, and how all of it stays maintainable two years from now.

In a small team one person wears both hats. In a large one, they are two jobs. Either way, the shift is from "I deliver tests" to "I make the team able to deliver good tests".

What actually changes in the work

Test strategy. Strategy starts with risk: identify what could hurt the product, decide what to test first, decide what not to automate, and split the effort between unit, API and end-to-end tests. A good strategy fits on a few pages and can be defended in front of a project manager. It says why, not just what.

Test architecture. An automation project that grows without a plan gets slow and brittle. The architect sets the rules: repository layout, data and environment management, parallel execution, readable reports, and conventions the whole team follows. The goal is simple: a new tester can add a test without breaking the others.

Metrics. Measuring is for deciding, not for decorating a dashboard. A coverage percentage alone does not tell you whether the right risks are covered. A lead picks few indicators, explains what they show and what they hide, and uses them to make trade-offs: where to put effort, when to stop, what to fix in the process.

Leading the team. Sharing out work, reviewing, coaching, pushing back on an unrealistic deadline, presenting a risk to the Product Owner: a growing share of the job happens in conversation. For a tester used to working alone on their scenarios, this is often the biggest change.

What US labor data says

The most solid public source is the BLS Occupational Outlook Handbook, which treats "software quality assurance analysts and testers" (SOC 15-1253) as its own occupation alongside software developers. Its page was last modified on August 27, 2026:

Jobs: about 187,600 software QA analysts and testers in 2025, against about 1,717,800 software developers. By simple division, that is roughly one QA analyst or tester for every nine developers.
Growth: projected employment growth from 2025 to 2035 is 6% for QA analysts and testers and 10% for software developers.
Pay: median annual pay in May 2025 was $104,300 for QA analysts and testers and $135,980 for software developers.

Two honest limits. First, the BLS does not publish figures for QA leads or test architects, so these numbers describe the occupation as a whole, not the step up. Second, a 6% projection is moderate growth, and it says nothing about your own chances of getting a given job.

My interpretation, which is an opinion and not a statistic: when there are relatively few tester seats per team, broadening your scope towards strategy, architecture and metrics is one way to become harder to replace by a script or an AI assistant. It is not the only way, and it does not suit everyone.

A one-week test before you decide

For a week, note how many of your questions at work are about how to write a test, and how many are about what to test and why. The second kind is the lead's territory. If most of your questions already sit there, the step is probably natural. If they do not, that is useful to know too.

If you are already asked "what is our test strategy?" and want an outside view of the answer, ADC offers a QA audit: a risk and coverage map of a team's testing, with a costed action plan.

What about certification?

ISTQB offers advanced levels. The CTAL-TM (Test Management, version 3.0) covers, among other topics, project test strategy, risk-based testing, test metrics, estimation, defect management and stakeholder relationships. It requires the Foundation level (CTFL) and enough practical experience, with the exact criteria set by ISTQB's national member boards. It is not mandatory, and it is not part of the specialisation below.

ADC's Lead QA & Architect specialisation

The track is for an experienced tester who wants to move towards team leadership, test strategy and test architecture. As shown on the specialisation page:

399 hours over 3 months, 100% online;
next session announced for January 2027 (confirm the exact date with an advisor);
one-to-one mentoring, learning platform for 6 months, session replays for 24 months;
topics: test strategy, team leadership, test architecture, metrics.

Launch price for individuals: €1,750 incl. VAT, payable in 1 to 4 interest-free instalments; for companies, €7,500 incl. VAT. Prerequisites are clarified with an advisor, since this track is not meant for beginners. ADC is a French-language online training provider, so check the teaching language before you enrol. No certification is promised for this specialisation, and no job, promotion or salary either.

Over to you

If you have made the jump from tester to lead, what was the first thing that surprised you: the strategy work, the metrics, or the conversations? And if you decided to stay hands-on instead, what made you choose that?

Sources: US Bureau of Labor Statistics: Software Developers, Quality Assurance Analysts, and Testers (Occupational Outlook Handbook, page last modified August 27, 2026) · ISTQB: Advanced Level Test Management CTAL-TM v3.0 (page modified August 22, 2026) · ADC: QA specialisation tracks (Lead QA & Architect)

Originally published on AutomationDataCamp. Written with AI assistance. Every figure comes from the sources listed above, as cited in the original article (checked on 19 October 2026).

AutomationDataCamp is an online software testing academy (Playwright, API testing, CI, AI-assisted testing, ISTQB prep). More on our site.

Testing now lives in the pipeline: what CNCF, DORA and BLS data mean for QA

AutomationDataCamp — Wed, 21 Oct 2026 13:00:00 +0000

A few years ago, a tester received a build, tested it and gave a verdict. Today code goes through a pipeline that builds, tests and ships on every change. A tester who cannot read that pipeline only sees part of the work. Here is what recent US-based studies say about it, with their limits.

TL;DR

Testing moves into the pipeline: it runs on every commit, not once at the end of a project.
Containers are mainstream: 82% of container users run Kubernetes in production (CNCF, January 2026).
The main obstacle is human: the most-cited challenge is cultural change with development teams (47%), not technology.
No figure measures "the market for DevOps testers", and none is claimed here.

What changes: testing is no longer "the last step"

In a team that ships often, nobody waits for a tester to open a spreadsheet of test cases. Automated checks start on every code change, and the pipeline stops when a test fails. The tester gains a more visible role: deciding which tests run, when, in which environment, and how to read a failure.

Employers in France already ask for it. In our QA skills barometer of 2 October 2026, CI/CD appears in 35.1% of the 405 QA job ads we analysed. That is a French sample, not a US one, but it points the same way. For concrete examples with Jenkins, GitLab CI and GitHub Actions, see test automation in CI/CD.

What the US-facing data says, and its limits

Each of these figures measures something specific.

Kubernetes in production. In the CNCF Annual Cloud Native Survey 2025, published on January 20, 2026, 82% of container users run Kubernetes in production, up from 66% in 2023. In the same survey, 98% of organisations say they have adopted cloud native techniques and 59% say much or nearly all of their development and deployment is cloud native.
A cultural obstacle first. The same survey names "cultural changes with the development team" as the top challenge (47%). CNCF says it is the first time the main challenge is organisational rather than technical. For a tester, being able to work with developers weighs as much as knowing the tools.
AI speeds up code, not trust. The 2025 DORA report from Google Cloud (September 23, 2025, nearly 5,000 technology professionals surveyed) says AI adoption among software professionals reached 90%, and is now linked to higher software delivery throughput. It also notes that the ongoing challenge is making sure software works as intended before it reaches users. That is exactly where testing sits.
Jobs. The US Bureau of Labor Statistics projects employment of software developers, QA analysts and testers to grow 10% from 2025 to 2035. For the QA analysts and testers line alone, it counts 187,600 jobs in 2025 and projects 198,200 in 2035, a 6% increase. The BLS groups these roles with developers for pay and does not break out DevOps testing, so I quote no salary here.

CI/CD, Docker, Kubernetes: seen from the test side

Four simple ideas, without needless jargon.

CI/CD: tests on every commit. A pipeline runs tests automatically when a change arrives. The tester chooses what runs on every commit (fast), on every merge (broader) and overnight (long). A flaky test that fails the pipeline for no reason costs the whole team its trust, so the job is as much about making tests reliable as automating them. If that sounds familiar, our earlier piece on flaky Playwright tests goes through the usual causes.

Docker: the same environment everywhere. A container packages the application and its dependencies. For a tester it ends "it works on my machine": you start the same environment locally and in the pipeline, with controlled test data.

Kubernetes: knowing where tests run. When the application runs on Kubernetes, test environments are created and destroyed there. You do not need to administer a cluster. You need to understand enough to run your tests in the right place and to read a log when something does not start.

Observability: seeing what happens after release. Logs, metrics and traces help explain a failure and spot a regression the tests missed.

A practical first step

If your tests only run on your laptop, move one smoke suite into a pipeline and make it block the merge when it fails. That single change forces every question that matters: where does the environment come from, where does the test data come from, and what happens when a test is flaky. If you want a second pair of eyes on how your tests fit into delivery, ADC also does a Cypress to Playwright migration with both suites running in CI during the transition.

The DevOps QA specialisation, briefly

ADC's DevOps QA specialisation covers CI/CD, Docker, Kubernetes and observability. The programme page lists 399 hours over 3 months, 100% online, with one-on-one mentoring, 6 months on the learning platform and 24 months of replays. The next session is announced for December; exact dates are confirmed with an advisor.

Two honest points. The programme pages state that ADC courses are taught in French, so ask an advisor about the language before you decide. And no certification is promised for this specialisation. Launch price: €1,500 for individuals (1 to 4 interest-free instalments), €7,500 for companies. Public funding schemes are not yet available at ADC. No programme guarantees a job or a salary.

Over to you

Where does your test suite run today: on every commit, on merge, nightly, or only when someone remembers? And in your team, is the harder part the tooling or, as CNCF's 47% suggests, the conversation with developers?

Sources: CNCF: Annual Cloud Native Survey 2025 (January 20, 2026) · Google: 2025 DORA report (September 23, 2025) · US Bureau of Labor Statistics: Software Developers, Quality Assurance Analysts, and Testers (projections 2025-35, page updated August 27, 2026) · ADC: QA Skills Barometer France (October 2, 2026)

Originally published on AutomationDataCamp. Written with AI assistance. Every figure comes from the sources listed above, as cited in the original article (checked on 19 October 2026).

AutomationDataCamp is an online software testing academy (Playwright, API testing, CI, AI-assisted testing, ISTQB prep). More on our site.

Is software testing still a good career in 2026? What US data says about QA and AI

AutomationDataCamp — Tue, 20 Oct 2026 13:00:00 +0000

"Will AI take my testing job?" is the question I hear most from people thinking about a career in QA. Instead of guessing, I went to the most reliable public source for US jobs, the Bureau of Labor Statistics (BLS), and read what it actually says about software QA analysts and testers, and about AI.

TL;DR

The outlook is positive: the BLS projects 6% growth for software QA analysts and testers from 2025 to 2035.
The pay data is solid: the median annual wage for this occupation was $104,300 in May 2025.
AI is a demand driver, not only a threat: the BLS links demand to software built for AI, IoT, robotics and other automation.
The work moves: drafting a first test gets cheaper with AI assistants; deciding what to test, reviewing the output and judging risk do not.

What the US data says about testers

The Occupational Outlook Handbook, last modified on 27 August 2026, gives these figures for software quality assurance analysts and testers:

About 187,600 jobs in 2025.
Projected employment growth of 6% from 2025 to 2035.
A median annual wage of $104,300 in May 2025.

For the wider group that includes software developers, QA analysts and testers, the BLS projects 10% growth over the same period and about 106,100 openings per year, mostly from workers who leave the occupation and need to be replaced. Read that carefully: 6% is the figure for testers alone, and the openings number covers the whole group, not testers only.

Also keep in mind that these are US national figures. Pay depends on the city, the sector, the seniority and the employer, and a median is not a starting salary. These numbers describe the market. They do not predict what anyone would earn.

What the BLS says about AI

The same page does not describe AI as shrinking this occupation. It says demand should stay strong because of the growth of software for AI, the Internet of Things, robotics and other automation, and that more investment in security software could raise demand for QA analysts and testers.

A fair reading: more software means more things to verify. The work moves, though. Writing a first draft of a test is getting cheaper with AI assistants. Deciding what to test, reviewing what the assistant produced and judging a risk are not getting cheaper.

I looked at the job-ad side too. In our QA Skills Barometer, a snapshot of 405 QA job ads in France taken on 2 October 2026, Playwright appears in 24.2% of ads and AI in 20.7%, with Jira and agile practices each in a larger share. That is a French sample, not a US one, but the direction is useful: automation and AI are on the list, and the basics of test management are still asked for more often. For a wider view, see AI testing trends in 2026.

The skills that travel

Whether a team is in Austin, Montreal or Lyon, the core of the job looks the same:

Test design: equivalence classes, boundary values, risk analysis. This is what makes automated tests worth having.
Automation: Playwright with TypeScript is a common modern stack for web and API tests. If you are on the hiring side and want that kind of suite built in your own repository instead, ADC also offers Playwright test automation services.
CI: running tests on every change, for example with GitHub Actions, and reading the failures.
Working with AI: generate, review, run and verify, and never accept a test you have not read.
Certification as a common language: ISTQB certificates are international.

A habit that costs nothing: the next time an AI assistant writes a test for you, read it line by line and ask what bug it would actually catch. If you cannot answer, the test is decoration.

One course, with honest limits

Our QA Automation & AI Tester course is built around exactly that list, starting from zero: testing fundamentals, then Playwright and TypeScript for web and API tests, tests in GitHub Actions, and generative AI applied to testing.

399 hours over 12 weeks, with 160 hours of live classes with a mentor and about 239 hours of self-paced work.
100% online, evening classes Monday to Friday, 19:30 to 21:30 Paris time.
ISTQB CTFL v4.0 and CT-GenAI v1.1 exams included, with a second attempt offered for each. The course is "certifying" in this precise sense: an ADC certificate awarded after a defence in front of a jury, plus the ISTQB exams.

The limits matter for a US reader. Classes are in French, so you need fluent French and the ability to read technical English. If you are not a French speaker, this course is not the right fit today. The evening slot is Paris time, early afternoon on the US East Coast. The next session runs from 15 October 2026 to 15 January 2027 with 12 places, and enrolment stays open until 22 October.

Price: €1,500 launch price for individuals (1 to 4 interest-free instalments), €7,500 for companies, both including VAT. French public funding such as the CPF, France Travail or an OPCO is not available yet. There is no promise of a job, a salary or that a recruiter will recognise the certificate.

Over to you

If you moved into QA in the last couple of years, which skill turned out to matter more than you expected: test design, automation or working with an AI assistant? And if you hire testers, what do you now ask candidates about AI that you did not ask two years ago?

Sources: US Bureau of Labor Statistics: Software Developers, Quality Assurance Analysts, and Testers (last modified 27 August 2026) · ADC: QA Skills Barometer (snapshot of 2 October 2026, France) · ISTQB

Originally published on AutomationDataCamp. Written with AI assistance. Every figure comes from the sources listed above, as cited in the original article (checked on 19 October 2026).

AutomationDataCamp is an online software testing academy (Playwright, API testing, CI, AI-assisted testing, ISTQB prep). More on our site.

Flaky Playwright tests: 8 common causes and how to fix each one

AutomationDataCamp — Fri, 09 Oct 2026 13:00:00 +0000

A flaky test fails, then passes on the next run with no code change. Playwright has a name for it: a test that “failed on the first run, but passed when retried” is reported as flaky. Retries hide the problem, they do not fix it. Below are eight common causes, how to spot each one, and the fix the Playwright documentation recommends, including two features from the latest releases: test locks (1.63) and --shuffle (1.64).

TL;DR

Never wait for time: Playwright’s own docs say tests that use waitForTimeout are “inherently flaky”.
Await the assertion, not the value: await expect(locator).toBeVisible() retries, expect(await locator.isVisible()) does not.
Order-dependent tests can now be found with npx playwright test --shuffle (Playwright 1.64).
Tests that share one resource can declare a named lock so they never run at the same time (Playwright 1.63).
Measure before you fix: --repeat-each to reproduce, traces on the first retry to understand, --fail-on-flaky-tests to stop new ones.

1. Hard waits

Symptom: the test passes on your laptop and fails on a slower CI runner, or the reverse.

Cause: page.waitForTimeout(2000) guesses how long the app needs. The Playwright API reference marks it as discouraged: “Never wait for timeout in production. Tests that wait for time are inherently flaky.”

Fix: wait for a state, not a duration. Locator actions already wait: before a click, Playwright checks that the element is visible, stable (“the same bounding box for at least two consecutive animation frames”), receives events and is enabled. Then assert the result with a web-first assertion such as await expect(page.getByRole('status')).toHaveText('Saved').

The same goes for waitForLoadState('networkidle'), which the docs also mark as discouraged: “Don't use this method for testing, rely on web assertions to assess readiness instead.”

2. Assertions that do not retry

Symptom: an assertion fails, but the screenshot shows the expected element.

Cause: the value was read once, before the page was ready. The Playwright best practices page gives this exact anti-pattern: expect(await page.getByText('welcome').isVisible()).toBe(true).

Fix: await expect(page.getByText('welcome')).toBeVisible(). Web-first assertions wait “until the expected condition is met”. A quick search for expect(await in your test folder often finds several of these.

3. Locators tied to the DOM structure

Symptom: the test breaks after a harmless markup change, or clicks the wrong one of two matching elements.

Cause: long CSS or XPath selectors. The docs warn: “Your DOM can easily change so having your tests depend on your DOM structure can lead to failing tests.”

Fix: prefer user-facing locators (getByRole, getByLabel, getByTestId). When a hidden copy of an element exists (a mobile menu, a template), Playwright 1.63 added locator.visible(), which “returns a locator that matches only visible elements”, for example page.locator('button').visible().

4. Tests that depend on each other

Symptom: a test passes in the full suite but fails when run alone, or the reverse.

Cause: one test creates data or state that another one silently needs. Playwright already gives each test a fresh browser context (“Playwright creates a context for each test”), so cookies and storage are not shared. But data in your backend is.

Fix: each test creates what it needs, for example through an API call in a fixture. To find the hidden dependencies, Playwright 1.64 added npx playwright test --shuffle, which “schedules tests in a random order, which helps to find tests that accidentally depend on each other”. The run prints a seed: pass it back (--shuffle 271828182 in the docs) to replay the same order while you debug.

5. Parallel tests fighting over one resource

Symptom: failures only appear with several workers, and never on the same test twice.

Cause: by default, Playwright runs test files in parallel in separate worker processes. Two tests that edit the same account setting or the same record overwrite each other.

Fix: first, give each test its own data (unique names, one user per worker). When the resource really is shared, Playwright 1.63 added test locks: “Tests that share a lock name never run concurrently, across files, workers and projects, while everything else keeps running in parallel.” You declare it with test('update user settings', { lock: 'user-settings' }, async ({ page }) => { ... }), or for a whole file with the lock option of test.describe.configure(). This is more precise than switching the whole file to serial mode.

6. Calls to services you do not control

Symptom: failures come in waves, on every branch at once.

Cause: a third-party API, payment sandbox or analytics script is slow or down. Your test is measuring someone else’s uptime.

Fix: the best practices page says to use “the Playwright Network API and guarantee the response needed”: page.route() with route.fulfill() returns a fixed response. Keep a few separate, clearly labelled tests that hit the real service.

7. Time, time zone and locale

Symptom: the test fails around midnight, at the end of the month, or only on the CI machine.

Cause: the app formats “today” or a price with the machine’s clock, time zone and language, and those differ between your laptop and CI.

Fix: set them. In the config, use: { locale: 'en-GB', timezoneId: 'Europe/Paris' } (the example from the emulation docs). For the date itself, the Clock API controls time: page.clock.setFixedTime() fixes Date.now() and new Date(), and page.clock.install() lets you pause or fast-forward timers.

8. A CI machine under too much load

Symptom: unrelated tests time out together, close to the timeout limit, and all pass when re-run alone.

Cause: too many workers for the machine, combined with a tight timeout. When workers is not set, Playwright uses 50% of the logical CPU cores. We hit this on our own website’s test suite: 5 tests failed in the full run, each running for 11 to 18 seconds with a 15-second test timeout. With --workers=1, all 89 tests passed. Nothing was wrong with the site.

Fix: set workers explicitly for CI, and do not lower timeouts to make the suite look faster. Before reporting a bug from a full run, re-run the failing test alone.

How to measure flakiness before fixing it

Reproduce: npx playwright test my.spec.ts --repeat-each=20 runs each test 20 times. A test that passes 20 times in a row is not proof, but one failure is.
Record evidence: the default config records a trace on-first-retry. For CI failures, the docs recommend “the Playwright trace viewer instead of videos and screenshots”: it shows the DOM, network and console at each step.
Read the report: with retries enabled, the report sorts tests into passed, flaky and failed. Track the flaky list like a bug list.
Stop new ones: --fail-on-flaky-tests makes the run “fail if any test is flagged as flaky”. Useful on a pull request, once the existing list is under control.

Can AI help?

AI agents can read a trace and suggest a fix, and Playwright now ships a healer agent for broken tests. But most of the causes above are design decisions: what data a test owns, which service is mocked, which resource is shared. An agent can propose a waitForTimeout just as easily as a web-first assertion, so review its fix with the same checklist.

Over to you

Which of these eight causes shows up most in your suite, and which one took you the longest to track down? If you have tried --shuffle or test locks since they shipped, I'd like to hear what they found.

Sources: Playwright docs: Retries (passed, flaky, failed) · Playwright docs: page.waitForTimeout (discouraged) · Playwright docs: Auto-waiting · Playwright docs: Best practices · Playwright docs: Isolation · Playwright docs: Parallelism · Playwright release notes (1.63 test locks, 1.64 --shuffle) · Playwright docs: Clock · Playwright docs: Emulation (locale, timezone) · Playwright docs: Command line (--repeat-each, --fail-on-flaky-tests) · Playwright docs: Trace viewer · Playwright docs: Test agents (planner, generator, healer)

Originally published on AutomationDataCamp. Written with AI assistance and reviewed by a test automation engineer; every Playwright option and quote was checked against the official documentation on 8 October 2026 (Playwright 1.64).

AutomationDataCamp is an online software testing academy (Playwright, API testing, CI, AI-assisted testing, ISTQB prep). More on our site.

How to Prepare for a QA Interview in 2026: 5 Questions You Will Be Asked

AutomationDataCamp — Thu, 08 Oct 2026 00:27:29 +0000

Candidates who know every definition by heart can still struggle in a QA interview, because the questions rarely test definitions. They test how you think.

Below are five questions that come up again and again for junior and mid-level testers, what the interviewer is actually looking for, and how to build an answer. They are not scripts. Read them, close the tab, and write your own version with an example from your own work or practice projects.

TL;DR

Answer with reasoning, not vocabulary. Interviewers want to hear how you decide, not a textbook quote.
Use risk to prioritize, and say out loud what you did not test.
A bug report is judged on one thing: can a developer reproduce it on the first try?
For "how would you test X?", ask questions first, then structure your ideas.
For automation, show judgment about what not to automate.

A quick word on how QA interviews are built

Most hiring processes for testers combine some of the same building blocks: a short screening call, a technical interview, a practical exercise (live or take-home), and a behavioral interview. The names and order change from one company to the next, but each round looks for something different. The five questions below are mostly technical-interview material, and the fourth one often turns into the practical exercise.

1. "Time is short. What do you test first?"

What they are checking: prioritization. Nobody tests everything, so the interviewer wants to see how you choose.

How to answer: talk about risk. Look at what changed in this release, what users rely on most, and what would hurt the business if it broke. Test the main happy paths of those areas first, then the most likely failure cases.

Then add the part most candidates forget: tell the team what you did not get to. A release decision made with a clear list of untested areas is a good decision, even if the coverage is incomplete.

"I'd start from what changed and what users depend on most, check the main flows there, then the likely failures. Before the release, I'd share exactly what I didn't cover, so the team decides with full information."

2. "What should a good bug report contain?"

What they are checking: clear communication. A vague report costs a developer time and costs you credibility.

How to answer: list the parts, then give the goal behind them.

a title that says what is wrong and where;
the environment: build or version, browser or device, operating system;
exact steps to reproduce;
expected result and actual result;
severity, with a one-line reason;
evidence: screenshot, short video, logs, the failing network request.

One bug per report. The goal: a developer reproduces it on the first try without having to ask you anything. If you can, mention a bug you reported and what made the report effective.

3. "A developer says: 'It works on my machine.' What do you do?"

What they are checking: collaboration under mild friction. Are you defensive, or curious?

How to answer: stay curious. Reproduce it again on your side, then compare the two environments: build version, browser, test data, user permissions, configuration, cached data, feature flags. Share the exact steps and the evidence, and offer to reproduce it together.

The point worth saying out loud: the difference between the two environments is often the clue to the bug itself. That turns a disagreement into a joint investigation, which is exactly what a team wants from its tester.

4. "How would you test this login page?"

What they are checking: structured thinking. This question (or "how would you test a pen / an elevator / a coffee machine?") is less about the object than about how you organize your ideas.

How to answer: start with questions. Who uses it? Is there "remember me", a lockout policy, single sign-on, two-factor authentication? What are the password rules? Then group your ideas instead of listing them at random:

Happy path: valid credentials log the user in and land on the right page.
Negative cases: wrong password, unknown email, empty fields, expired account.
Boundaries: minimum and maximum length, spaces, special characters, copy-paste.
Security: password is masked, lockout after repeated failures, error messages don't reveal whether the email exists.
Usability and accessibility: keyboard navigation, labels read by a screen reader, clear error messages.
Compatibility: the main browsers and a mobile viewport.

Finish by saying which of these you would run first, and why. The structure of your answer matters more than its length.

5. "What would you automate first?"

What they are checking: judgment. Automation is a cost as well as a benefit, and mid-level candidates in particular are expected to know the difference.

How to answer: automate what is stable, repetitive and valuable: smoke tests, core user journeys, regression checks, and API tests with many data combinations. Keep exploratory testing, usability judgment, one-off checks and features that still change every sprint as manual work, at least for now.

If the role mentions a tool, connect your answer to it. With Playwright, for example, you can mention that the official best practices recommend testing user-visible behavior, using locators, and keeping tests isolated from one another. Be honest about your level: "I've used it on a personal project, here is what I built" beats bluffing every time.

API questions often follow this one. A classic: what is the difference between 401 and 403? A 401 means the request lacks valid authentication; a 403 means the server understood who you are but refuses access.

Tool expectations also vary by market. When we counted the skills in 405 QA job ads in France in October 2026, Playwright appeared in 24.2% of them and Selenium in 21.7%. Read the job ad closely and prepare for the tools it names.

How to practice in the week before

Write your answers down, then say them out loud with a timer. Most answers should take between 45 and 90 seconds.
Prepare four to six real stories in STAR format (Situation, Task, Action, Result) for behavioral questions. Say "I", not "we", and keep the Action part the longest.
Redo one practical exercise: write test cases for a login form, or a bug report for a bug you found in any app you use.
Prepare two or three questions for them, for example: "Who decides when a release is ready?"

If you want a structured workbook

I put all of this, and more, into a printable and fillable PDF: the Software Testing Interview Workbook (36 pages, in English). It covers 30 common questions with model answers, 13 core testing concepts, three practical exercises with worked solutions (login test cases, a checkout spec review, a bug report), automation and API basics, STAR worksheets and a 7-day prep plan. It's a paid product on Etsy ($10.99): Software Testing Interview Workbook. Everything in this article is free to use without it.

Over to you

Which interview question caught you off guard the most, and how would you answer it today? Share it in the comments; I'll reply to each one.

Sources: Playwright — Best Practices · MDN — 401 Unauthorized · MDN — 403 Forbidden · AutomationDataCamp — QA Skills Barometer, October 2026

Written with AI assistance and reviewed by a test automation engineer; links were checked on 7 October 2026. The interview advice reflects our own experience, not a survey.

AutomationDataCamp is an online software testing academy (Playwright, API testing, CI, AI-assisted testing, ISTQB prep). More on our site.

How to test an LLM chatbot or a RAG app in 2026: a tester's checklist

AutomationDataCamp — Tue, 06 Oct 2026 13:00:07 +0000

More and more teams ship a chatbot or a “chat with your documents” feature, and testers are asked to sign it off. The usual tools still apply, but the method changes: the same question can get two different answers, the answer depends on documents the model retrieved, and the user can try to talk the model into misbehaving. Here is how we approach it, with the public references behind each check.

TL;DR

Don’t assert exact text: assert properties (required facts, refusal, format, sources) and run each case several times.
Test retrieval and generation separately: a wrong answer often comes from the wrong documents, not from the model.
Use the OWASP Top 10 for LLM Applications (2025) as a security checklist: prompt injection, sensitive information disclosure, system prompt leakage, excessive agency…
Check the AI Act transparency rule: since 2 August 2026, users must be told they are interacting with an AI system.
Tools exist: promptfoo (open-source evals and red teaming), Ragas (RAG metrics), and Playwright for the UI.

Why do classic assertions break on a chatbot?

Because the output is not deterministic. Ask the same question twice and you may get two correct answers worded differently, or one correct and one wrong. An assertion like expect(answer).toBe("Your order ships in 3 days") fails on a good answer and tells you nothing about a bad one.

What works better is to assert properties of the answer:

it contains the required facts (the delay, the price, the right product name);
it does not contain forbidden content (another customer’s data, an internal URL, a competitor’s price);
it refuses what is out of scope, instead of inventing;
it respects the expected format (JSON schema, length, language);
it cites its sources when the product promises it.

Then run each case several times and track a pass rate, not a single green or red. A case that passes 7 times out of 10 is a finding, not a pass.

How do you test a RAG application?

A RAG (retrieval-augmented generation) app first retrieves documents, then generates an answer from them. Test the two steps separately, otherwise you cannot tell where a wrong answer comes from.

Retrieval: for a set of questions with known answers, check that the right passages come back. Ragas calls this context recall: how many of the relevant documents were successfully retrieved.
Generation: check that the answer sticks to what was retrieved. Ragas calls this faithfulness: how factually consistent the response is with the retrieved context, on a 0 to 1 scale.

Build a small reference set first: 30 to 50 real questions, with the expected answer and the document it should come from. It becomes your regression suite every time someone changes the prompt, the model or the chunking.

Which security tests come from the OWASP LLM Top 10?

The OWASP Top 10 for LLM Applications (2025) is the most used public checklist. Several entries translate directly into test cases:

LLM01 Prompt Injection: user input that alters the model’s behaviour. Test it directly (“ignore your instructions and…”) and indirectly, by putting hidden instructions in a document or web page the app will read.
LLM02 Sensitive Information Disclosure: try to obtain another user’s data, keys or internal information.
LLM05 Improper Output Handling: check what happens when the model’s output is rendered or passed on, for example HTML or script in an answer displayed in the page.
LLM06 Excessive Agency: if the bot can call tools (send an email, cancel an order), check it cannot do more than the user is allowed to.
LLM07 System Prompt Leakage: ask for the instructions, in several languages and phrasings.
LLM09 Misinformation: questions with no answer in the documents; the right behaviour is to say so.
LLM10 Unbounded Consumption: very long inputs, rapid repeated requests; check limits and costs.

What does the AI Act add for a chatbot?

A test case that is easy to forget. According to the European Commission, the transparency obligations of Article 50 apply since 2 August 2026: among other things, people must be informed that they are interacting with an AI system. Check that the disclosure is there, visible, and in the user’s language. (The high-risk rules are a separate matter: since the AI Omnibus they apply from 2 December 2027.)

Which tools help?

promptfoo: an open-source CLI and library for evaluating and red-teaming LLM apps, usable in CI (a GitHub Action exists). Good for running your reference set and attack prompts on every change.
Ragas: RAG metrics such as context precision, context recall, faithfulness and response relevancy.
Playwright: for the chat interface itself (streaming, errors, disclosure banner, accessibility), as for any web UI.

Tools score; they do not decide. A human still reviews the failing cases and the thresholds.

A starter checklist

A reference set of 30 to 50 real questions with expected facts and source documents.
Each case run several times, with a target pass rate.
Retrieval checked on its own (right passages returned).
Answers checked against the retrieved context (no invented facts).
Out-of-scope and unanswerable questions: the bot says it does not know.
Direct and indirect prompt injection attempts.
System prompt and other users’ data cannot be extracted.
Tool calls limited to what the user is allowed to do.
Model output safely rendered in the page.
AI disclosure visible to the user.

Over to you

How do you test your chatbot today? Pass rates, golden datasets, red-teaming prompts, or still mostly by hand? I'd love to compare approaches in the comments.

Sources: OWASP — Top 10 for LLM Applications (2025) · OWASP — LLM01:2025 Prompt Injection · Ragas — Available metrics · promptfoo — Introduction · European Commission — Navigating the AI Act

Originally published on AutomationDataCamp. Written with AI assistance; definitions and dates were checked against the sources listed above on 5 October 2026.

AutomationDataCamp is an online software testing academy (Playwright, API testing, CI, AI-assisted testing, ISTQB prep). More on our site.

AI in software testing in 2026: 5 changes that are actually shipping (not hype)

AutomationDataCamp — Mon, 05 Oct 2026 14:35:21 +0000

Every week brings a new “AI will replace testers” headline. We looked instead at what testers can actually use in 2026: features in Playwright’s release notes, ISTQB syllabi, job ads and EU law. Five changes stand out, and each one says something about what a tester’s job looks like now.

TL;DR

Agents now ship inside the test framework: Playwright Test Agents (planner, generator, healer) since version 1.56.
Playwright is built for coding agents: CLI debugging and trace analysis for agents (1.59), MCP server bundled (1.62).
ISTQB split “testing AI” from “testing with AI”: CT-AI v2.0 (April 2026) and CT-GenAI v1.1.
Employers ask for it: one QA job ad in five in France mentions AI.
The AI Act high-risk deadline moved to 2 December 2027 (Annex III), after the AI Omnibus entered into force on 27 July 2026.

1. Are AI agents now part of the test framework itself?

Yes, in Playwright at least. Since version 1.56 (October 2025), Playwright ships Playwright Test Agents, three agent definitions that guide a language model through building a test:

planner: explores the app and produces a Markdown test plan;
generator: turns that plan into Playwright Test files;
healer: runs the suite and automatically repairs failing tests.

One command, npx playwright init-agents, generates the agent definitions for your assistant (Claude Code, VS Code and others). The shift is important: AI assistance is no longer a third-party plugin bolted onto the framework, it is part of the framework’s own documentation and tooling.

2. What has Playwright added for coding agents in 2026?

Two releases made Playwright easier for an agent to drive and debug:

1.59 (April 2026): coding agents can run npx playwright test --debug=cli to attach and debug tests over playwright-cli, and npx playwright trace to explore a trace and understand failing or flaky tests from the command line.
1.62 (July 2026): Playwright now bundles the Playwright MCP server and playwright-cli, runnable with npx playwright mcp and npx playwright cli.

In practice, the loop “a test fails, the assistant reads the trace, proposes a fix” no longer needs custom glue code. Our Playwright MCP guide shows a first explore-then-generate scenario.

3. Why did ISTQB split its AI certifications?

Because they describe two different jobs. CT-AI v2.0, released on 17 April 2026, certifies the ability to test AI-based systems (data, models, generative AI products). CT-GenAI v1.1 certifies the ability to use generative AI to test (prompting, risks, LLM-powered tools). Since v2.0, “using AI for testing” belongs to CT-GenAI. Both require the CTFL. The English CT-AI v1.0 exam remains available until 21 April 2027. Details in our CT-AI vs CT-GenAI comparison.

4. Do employers really ask testers for AI skills?

Increasingly. In our count of 405 job ads with “QA” in the title published in France over 60 days (snapshot of 2 October 2026), 20.7% mention AI or generative AI. That word covers several realities: testing an AI-based product, using assistants to write tests, or simply the company’s context. For comparison, 24.2% mention Playwright and 40% mention Jira. The full figures are in our QA Skills Barometer.

5. What changed for the EU AI Act in 2026?

The calendar. The AI Omnibus, which amends the AI Act (Regulation 2024/1689), entered into force on 27 July 2026. According to the European Commission, rules for high-risk AI systems listed in Annex III now apply from 2 December 2027, and those for AI embedded in products covered by Annex I from 2 August 2028. For testers working on AI products, this is extra time to build risk management, data quality and robustness testing, not a reason to drop them.

What does it mean for a tester today?

The common thread: AI writes more tests, and humans review them. Before merging a test generated by an assistant, we check seven points:

a business assertion (not just “the page loaded”);
accessible locators (roles and names, not brittle CSS);
isolated test data;
no secrets in the prompt;
never run against production;
a limited exploration scope for the agent;
the test seen failing at least once, so you know it can catch a bug.

Test design (equivalence partitions, boundary values, risk-based prioritisation) matters more, not less: it is what you need to judge whether a generated test is worth keeping.

Over to you

Which of these have you actually used in a real project? Playwright's healer agent, MCP, an AI-generated test that made it to main? Tell me in the comments what worked, and what you had to throw away.

Sources: Playwright — Release notes (1.56, 1.59, 1.62) · Playwright — Test Agents · ISTQB — Certified Tester AI Testing (CT-AI) v2.0 · ISTQB — Testing with Generative AI (CT-GenAI) · European Commission — AI Omnibus enters into force · AutomationDataCamp — QA Skills Barometer, October 2026

Originally published on AutomationDataCamp. Written with AI assistance; every version number and date was checked against the sources listed above on 5 October 2026.

AutomationDataCamp is an online software testing academy (Playwright, API testing, CI, AI-assisted testing, ISTQB prep). More on our site.

Playwright now beats Selenium in French QA job ads: what 405 ads ask testers for

AutomationDataCamp — Mon, 05 Oct 2026 11:03:50 +0000

Which tools and skills do employers actually ask testers for? Rather than repeat the usual "top 10 testing tools" lists, we counted.

On 2 October 2026, we analysed job ads published in France over the previous 60 days: 405 ads with "QA" in the title and 1,180 ads with "test" or "testeur" in the title. Here is what they mention.

TL;DR

Playwright has caught up with Selenium: 24.2% of QA ads mention Playwright, versus 21.7% for Selenium.
Test management outweighs any automation tool: Jira (40%) and Xray (26.2%) come ahead of every framework.
AI is entering job descriptions: one QA ad in five (20.7%) mentions it.
Performance testing stays marginal: JMeter, k6 and Gatling each appear in less than 2% of QA ads.

Method in brief

We used the public API of Adzuna, an aggregator that collects ads from many job boards. For each skill, we counted ads published in France over the last 60 days whose title contains "QA" (or "test") and whose full text mentions the skill.

"QA" ads (405): QA engineer, QA automation, QA tester. The core of software testing, mostly web and SaaS.
"Test" ads (1,180): broader. They also include validation roles for embedded, railway or industrial systems, where web tools are less in demand, so shares are lower.

Automation tools: Playwright level with Selenium

Skill	"QA" ads	Share	"Test" ads	Share
Playwright	98	24.2%	81	6.9%
Selenium	88	21.7%	84	7.1%
Cypress	61	15.1%	54	4.6%
Robot Framework	36	8.9%	59	5%
Appium	21	5.2%	16	1.4%
UFT / QTP	4	1%	10	0.8%
TestComplete	2	0.5%	5	0.4%

In QA ads, Playwright comes first. In the more industrial "test" ads, Playwright and Selenium are neck and neck, and Robot Framework rises to 5%. Legacy proprietary tools stay under 1%.

Test management: Jira and Xray first

This is the clearest result.

Skill	"QA" ads	Share	"Test" ads	Share
Jira	162	40%	212	18%
Xray	106	26.2%	114	9.7%
Cucumber / Gherkin	57	14.1%	45	3.8%
Squash TM	34	8.4%	76	6.4%
TestRail	20	4.9%	14	1.2%
ALM / Quality Center	10	2.5%	58	4.9%

Writing and tracking test cases in a tool matters as much as automating them. Squash TM, an open-source tool widely used in France, shows up more than TestRail.

API and performance testing

Skill	"QA" ads	Share	"Test" ads	Share
Postman	81	20%	88	7.5%
REST Assured	62	15.3%	63	5.3%
SoapUI	9	2.2%	14	1.2%
JMeter	7	1.7%	10	0.8%
k6	4	1%	3	0.3%
Gatling	4	1%	3	0.3%

API testing is well established. Performance testing remains a specialty, rarely asked of generalist testers.

Languages: Python first

Skill	"QA" ads	Share	"Test" ads	Share
Python	74	18.3%	199	16.9%
SQL	60	14.8%	123	10.4%
Java	54	13.3%	104	8.8%
JavaScript / TypeScript	52	12.8%	47	4%
C# / .NET	18	4.4%	50	4.2%

SQL right behind Python is a useful reminder: querying a database to check a result is a daily task.

CI/CD: one QA ad in three

Skill	"QA" ads	Share	"Test" ads	Share
CI/CD	142	35.1%	178	15.1%
GitLab	59	14.6%	64	5.4%
Jenkins	46	11.4%	57	4.8%
Git	42	10.4%	78	6.6%
Docker	17	4.2%	19	1.6%
GitHub Actions	12	3%	12	1%

An automation tester is expected to run tests in the pipeline, not just on their own machine. In France, GitLab is well ahead of GitHub Actions.

Agile, AI and ISTQB

Skill	"QA" ads	Share	"Test" ads	Share
Agile / Scrum	197	48.6%	265	22.5%
AI / generative AI	84	20.7%	143	12.1%
ISTQB	59	14.6%	103	8.7%

"AI" covers testing AI-based products, using assistants to write tests, or simply company context. ISTQB is a recognised asset, rarely a hard requirement.

Where, which contract, which salary

On a sample of 605 unique test and QA ads:

Location: the Paris region (Île-de-France) accounts for 31.9% of ads, followed by Auvergne-Rhône-Alpes (12.6%), Provence-Alpes-Côte d'Azur (11.9%) and Occitanie (10.9%).
Contracts: 44.3% permanent (CDI), 15.2% fixed-term, assignment or freelance, 40.5% not specified.
Seniority: only 2.1% of titles explicitly say "junior"; 14.5% target a senior or lead profile.
Salaries: 229 ads (37.9%) display one. The median is €42,500 gross per year, half between €40,000 and €45,000. Senior and lead roles show a median of €50,000.

What it means if you are learning

Learn a modern framework: Playwright (or Selenium) remains the technical core of test automation.
Don't neglect test management tools: Jira, Xray or Squash TM are cited more often than any framework.
Run your tests in CI: one QA ad in three asks for it.
Get ready for AI: one QA ad in five mentions it, and reviewing what an assistant writes is becoming part of the job.

Limitations

Adzuna is not exhaustive, and an ad posted on several sites may count more than once.
A mentioned skill is not always required: it may be a "nice to have".
Titles are imperfect: a QA role with another title escapes both bases.
"AI" is a broad word.
This is a one-off snapshot. We will repeat it every quarter, starting in December 2026.

Over to you

Does this match your market? If you hire or work as a tester outside France (UK, Canada, US, Belgium, Switzerland…), I'd love to know in the comments whether Playwright has overtaken Selenium where you are, and which tool your last job ad asked for.

Originally published as the QA Skills Barometer (full FAQ and chart there; French version). You may reuse the figures with a link to the source: "QA Skills Barometer, AutomationDataCamp, October 2026". Job ad data: Adzuna. This article was written with AI assistance; the figures come from our own count and were checked against the original page.

AutomationDataCamp is an online software testing academy (Playwright, API testing, CI, AI-assisted testing, ISTQB prep). More on our site.

ISTQB CT-AI vs CT-GenAI: which AI certification should a tester take?

AutomationDataCamp — Fri, 02 Oct 2026 00:57:24 +0000

ISTQB offers two certifications around artificial intelligence, and their names are easy to mix up. CT-AI ("AI Testing") certifies the ability to test AI-based systems: data, machine learning models, generative AI systems. CT-GenAI ("Testing with Generative AI") certifies the ability to use generative AI to test: prompting, risk management, LLM-powered tools. CT-AI version 2.0, released on 17 April 2026, made that boundary clear. This guide compares both with official figures and helps you choose.

CT-AI = testing AI; CT-GenAI = testing with AI. Since CT-AI v2.0, "using AI for testing" belongs to CT-GenAI.
Same prerequisite: the CTFL (Foundation Level) certification.
Same exam format: 40 questions in 60 minutes; CT-AI v2.0 pass mark is 29 out of 44 points, CT-GenAI 30 out of 46.
CT-AI v1.0 is retiring: English exam until 21 April 2027, other languages until 21 October 2027; v1.0 certificates remain valid.

CT-AI vs CT-GenAI in one sentence

CT-AI is for testers whose product contains AI: a recommendation engine, a fraud detection model, an LLM-based chatbot. The question is "is this AI system reliable, fair, robust?". CT-GenAI is for testers who want to use generative AI in their own work: writing test cases, generating data, analysing results, building internal tools on top of an LLM. The question becomes "how do I use these assistants without lowering the quality of my tests?".

The two can complement each other. A tester can use an AI assistant (CT-GenAI) to test a product that itself embeds a model (CT-AI). But the skills assessed are different.

What does ISTQB CT-AI v2.0 cover?

The Certified Tester AI Testing v2.0 syllabus was released by the ISTQB General Assembly on 17 April 2026. It is a rewrite of the 2021 version 1.0: it goes from 11 to 7 chapters, and recommended training goes from four to three days. Content on using AI for testing was removed, with ISTQB pointing to CT-GenAI for that topic. The seven chapters, with the training time given in the syllabus:

Introduction to Artificial Intelligence (120 min)
Quality Characteristics for AI-Based Systems (45 min)
Machine Learning (375 min)
Testing AI-Based Systems (195 min), including a section on testing generative AI and large language models, with red teaming
Input Data Testing for Machine Learning Systems (180 min)
Model Testing for Machine Learning Systems (225 min)
Machine Learning Development Testing (30 min)

That is 1,170 minutes in total, about 19.5 hours. ISTQB's stated outcomes include understanding how neural networks work, knowing the ISO/IEC 25059 quality characteristics, calculating machine learning performance metrics, and designing and executing test cases for machine learning systems. Target audience: testers, test analysts, test engineers, test managers, consultants, data analysts, developers and UAT roles.

What does ISTQB CT-GenAI cover?

The Certified Tester – Testing with Generative AI syllabus was approved by the ISTQB General Assembly on 25 July 2025. The current version is 1.1, a minor update announced in April 2026 that, according to ISTQB, does not change the structure, learning objectives or scope. Its five chapters:

Introduction to Generative AI for Software Testing
Prompt Engineering for Effective Software Testing
Managing Risks of Generative AI in Software Testing (hallucinations, reasoning errors, bias, data privacy, energy consumption, regulation)
LLM-Powered Solutions for Software Testing
Deploying and Integrating Generative AI in Test Organizations

The v1.0 syllabus plans 815 minutes of training, about 13.5 hours, of which 365 minutes for prompt engineering alone: it is the practical core of the certification.

Side-by-side comparison

Criterion	CT-AI v2.0	CT-GenAI v1.1
Focus	Testing AI-based systems	Using generative AI to test
Prerequisite	CTFL	CTFL
Chapters	7	5
Training time in syllabus	1,170 min (about 3 days)	815 min (v1.0 syllabus)
Exam	40 questions, 60 min	40 questions, 60 min
Pass mark	29 out of 44 points	30 out of 46 points
Extra time	+25% for non-native speakers	+25% for non-native speakers
Typical profile	Tester of a product embedding ML or an LLM	Tester who wants to work with AI assistants

Sources: official ISTQB pages and syllabi, accessed on 2 October 2026 (list at the end).

Which one fits your profile?

You test a product that includes AI (scoring, recommendation, vision, chatbot, agent): CT-AI. You will learn to test data, models and generative AI systems, topics that classic test techniques do not cover.
You want to be more productive with ChatGPT, Claude or Copilot: CT-GenAI. It gives a method to write test prompts, check what the AI produces and manage its risks.
You are aiming for a test automation role: CT-GenAI is the most useful day to day, as coding assistants are now everyday tools (see our Playwright MCP guide). To go deeper into automation itself, ISTQB offers the advanced CTAL-TAE (Test Automation Engineering v2.0): it requires CTFL and sufficient practical experience, and its exam has 40 questions in 90 minutes, passed at 43 out of 66 points.
You are a test manager or consultant: both make sense. CT-GenAI covers the strategy for adopting generative AI in a test team, CT-AI the test strategy for an AI-based product.

CTFL first, in every case

CTFL v4.0 is the shared prerequisite. Its exam has 40 questions in 60 minutes and is passed at 26 out of 40 points. If you do not have it yet, start there. See all the certifications we prepare on our certifications page.

How to prepare with ADC

Our QA Automation & AI Tester programme, taught in French, includes preparation for the ISTQB CTFL and CT-GenAI exams, exams included. We do not prepare for CT-AI at this time. AutomationDataCamp is not an ISTQB-accredited training provider: we prepare candidates for the exams, which are delivered by official exam providers.

Frequently asked questions

What is the difference between ISTQB CT-AI and CT-GenAI?

CT-AI (AI Testing) teaches how to test AI-based systems: data, machine learning models and generative AI systems. CT-GenAI (Testing with Generative AI) teaches how to use generative AI to test: prompting, risks such as hallucinations, and LLM-powered tooling. Since CT-AI version 2.0, the 'using AI for testing' content was removed from CT-AI and points to CT-GenAI instead.

Do I need CTFL for CT-AI or CT-GenAI?

Yes. ISTQB requires the Certified Tester Foundation Level (CTFL) certification for both CT-AI v2.0 and CT-GenAI.

Is my CT-AI v1.0 certificate still valid?

Yes. According to ISTQB, certifications obtained under version 1.0 remain valid, with no requirement to retake the exam. The v1.0 exam stays available in English until 21 April 2027 and in other languages until 21 October 2027.

What are the CT-AI and CT-GenAI exams like?

Both have 40 questions in 60 minutes, with 25% extra time for candidates not taking the exam in their native language. CT-AI v2.0 is scored out of 44 points with a pass mark of 29; CT-GenAI out of 46 points with a pass mark of 30.

Which one should a beginner choose?

Start with CTFL, which is required for both. Then choose CT-GenAI if your goal is to be more productive with AI assistants day to day, and CT-AI if you test, or will test, a product that embeds machine learning or a language model.

Sources and references

Written with AI assistance; every figure was checked against the official ISTQB sources listed above, accessed on 2 October 2026. A French version is available: ISTQB CT-AI ou CT-GenAI.

Prepare for CTFL and CT-GenAI

Our programme prepares you for the ISTQB CTFL and CT-GenAI exams, exams included, with mock exams in the official format. Courses are taught in French. Programme details: automationdatacamp.com/en/formations.

Playwright MCP: let an AI assistant explore your app, then write the test (Claude Code, Copilot, Cursor)

AutomationDataCamp — Thu, 01 Oct 2026 23:36:52 +0000

Playwright MCP is Microsoft's official Model Context Protocol server for Playwright. It gives an AI assistant such as Claude Code, GitHub Copilot or Cursor the tools to drive a real browser: navigate, click, fill forms and read the page. The interesting part for testers: the assistant reads the page through structured accessibility snapshots, not screenshots, so no vision model is needed and it naturally "sees" the roles and accessible names that good Playwright locators are built on.

This post walks through installation, a first explore-then-generate scenario, how MCP compares with Playwright CLI and the Playwright Test Agents, and the rules we teach to keep AI-generated tests trustworthy.

Install it in one command

Node.js is the only prerequisite (the server runs with npx). The browser is headed by default, which is handy to watch what the assistant does.

Claude Code

claude mcp add playwright npx @playwright/mcp@latest

VS Code (GitHub Copilot agent mode)

code --add-mcp '{"name":"playwright","command":"npx","args":["@playwright/mcp@latest"]}'

Cursor and other MCP clients accept the standard JSON configuration:

{
  "mcpServers": {
    "playwright": {
      "command": "npx",
      "args": ["@playwright/mcp@latest"]
    }
  }
}

Options worth knowing (append them after @playwright/mcp@latest):

--headless: no visible browser (CI machines, containers)
--isolated: keep the profile in memory, start from a clean session every time
--storage-state <path>: start already logged in
--secrets <path>: a dotenv file, so credentials never appear in your prompt
--test-id-attribute: defaults to data-testid
--caps: optional capabilities such as testing (assertion and locator-generation tools), network (route mocking), storage, pdf, vision, devtools

A first scenario: explore, then generate

Step 1, exploration. Be precise in the prompt:

Using Playwright MCP, open http://localhost:3000/login.
Describe the fields, buttons and messages on the page.
Then try to log in with a wrong password and report the exact error message.
Do not submit any other form.

The assistant calls browser_navigate, browser_snapshot, browser_fill_form / browser_type and browser_click, then reads the page again. You get a report of the elements (with roles and accessible labels) and of the observed behaviour: a small, documented exploratory session.

Step 2, generation.

From what you observed, write tests/login.spec.ts with Playwright Test in TypeScript:
one successful login and one wrong-password case. Use getByRole and getByLabel only,
no CSS selectors, read credentials from environment variables.
Then run the tests and fix them if they fail.

A typical result (labels come from the demo app; yours will come from the real exploration):

import { test, expect } from '@playwright/test';

test.describe('Login', () => {
  test.beforeEach(async ({ page }) => {
    await page.goto('/login');
  });

  test('a valid user reaches the dashboard', async ({ page }) => {
    await page.getByLabel('Email').fill(process.env.TEST_USER_EMAIL!);
    await page.getByLabel('Password').fill(process.env.TEST_USER_PASSWORD!);
    await page.getByRole('button', { name: 'Sign in' }).click();

    await expect(page).toHaveURL(/\/dashboard/);
    await expect(page.getByRole('heading', { name: 'Dashboard' })).toBeVisible();
  });

  test('a wrong password shows an error', async ({ page }) => {
    await page.getByLabel('Email').fill(process.env.TEST_USER_EMAIL!);
    await page.getByLabel('Password').fill('wrong-password');
    await page.getByRole('button', { name: 'Sign in' }).click();

    await expect(page.getByRole('alert')).toContainText('Invalid credentials');
    await expect(page).toHaveURL(/\/login/);
  });
});

Step 3, human review. Read it like a colleague's pull request. Do the assertions check a business outcome or just that a click happened? Does the error case also check the user is not logged in? Is an edge case missing (empty field, locked account)? Run it yourself and make it fail once on purpose to prove it catches a regression.

MCP, CLI or Test Agents?

Approach	How it works	Best for
Playwright MCP	The AI calls `browser_*` tools and reads accessibility snapshots	Exploration, self-healing tests, long autonomous loops with a persistent browser context
Playwright CLI + Skills	The coding agent runs short commands described by Skills	Coding agents juggling a large codebase: more token-efficient
Playwright Test Agents (1.56+)	Three agents: planner, generator, healer	Building or maintaining a Playwright Test suite with an assistant

The Playwright MCP README is explicit: for coding agents, CLI + Skills is often the better choice because it avoids loading large tool schemas and verbose accessibility trees into the context. MCP stays relevant when the agent needs to reason over page structure with persistent state.

Test Agents are set up in an existing project:

npx playwright init-agents --loop=claude   # or --loop=vscode, --loop=opencode

For Claude Code this writes agent definitions in .claude/agents/ and a .mcp.json that starts npx playwright run-test-mcp-server. The planner explores the app and writes a Markdown test plan, the generator turns it into test files, the healer runs the suite and repairs failing tests (as a last resort it marks a test test.fixme() with a comment). Re-run the command after each Playwright upgrade.

Rules we teach for AI-generated tests

Review every generated line. A passing test is not necessarily a good test. The AI checks what it sees; the tester defines what the business expects.
Demand accessible locators (getByRole, getByLabel, getByTestId) and say so in the prompt.
Own your test data. Dedicated accounts and datasets, recreated before each run.
Keep secrets out of prompts and code: --secrets for exploration, environment variables for tests.
Never point the agent at production. An assistant that clicks on its own can submit, delete or pay. Use a test environment and --isolated.
Watch token cost. Every snapshot and tool description fills the context. Scope the exploration to one flow.
Keep CI deterministic. AI writes and repairs tests; plain Playwright Test runs them in the pipeline.

What it changes for testers

Less time writing selectors and boilerplate, more time on test design (equivalence partitions, boundary values, decision tables), test data and code review. You still need to read and fix TypeScript: you can't review what you don't understand.

This article is adapted from our French guide Playwright MCP : tester avec l'IA. It was written with AI assistance; commands and options were checked against the official Playwright MCP and Playwright documentation.

AutomationDataCamp is an online software testing academy (Playwright, API testing, CI, AI-assisted testing, ISTQB prep). A ready-to-run Playwright + TypeScript starter with CI is on GitHub; more on our site.

Sources: microsoft/playwright-mcp · microsoft/playwright-cli · Playwright Test Agents · Playwright locators