DEV Community: Antoine Dubois

AI-Assisted QA Changes the Testing Job, Not the Testing Need

Antoine Dubois — Thu, 04 Jun 2026 09:39:55 +0000

Internal note to the team, we need to improve test coverage and keep shipping, which means we should treat AI as a helper in the workflow, not as a replacement for testing discipline.

AI-assisted development changes the shape of our risk. It can produce more code faster, but it also increases the chance that small logic mistakes, brittle selectors, and shallow test cases slip through review. The answer is not to add more manual checking everywhere. The answer is to be more deliberate about what we review, what we automate, and where we let AI help.

What changes when AI writes part of the code

The first thing that changes is review. When a developer uses AI to draft a feature, a test, or a refactor, the reviewer is no longer only checking intent and style. The reviewer also needs to check whether the generated code matches the product rule, whether it introduced a hidden dependency, and whether it quietly weakened coverage.

That does not mean every AI-assisted change deserves extra ceremony. It means our review checklist should shift from "does this look correct" to "what did the model assume, and did we verify those assumptions?" That is especially important for test code, because generated tests often look plausible even when they do not prove much.

Coverage should move from volume to signal

AI tends to produce more test cases, but more cases are not the same as better coverage. If a generated test suite repeats the same happy path under slightly different names, the team gets a false sense of safety. Coverage should answer a more practical question, where are we most likely to break the user experience, and where will a test actually catch it?

For chat and other AI features, prompt-by-prompt manual checks are a trap. They do not scale, and they encourage a habit of eyeballing output instead of verifying behavior. A better pattern is to build assertions around expected properties, create eval sets for representative prompts, and add regression coverage for failure modes. The article How to Test AI Chat Features Without Relying on Prompt-by-Prompt Manual Checks is a useful practical reference here, because it focuses on assertions, guardrails, and repeatable checks instead of one-off spot checks.

Automation decisions need a maintenance lens

AI also affects what we choose to automate. It is tempting to let an assistant generate Playwright tests for every flow, then call it done. The hidden cost shows up later, when those tests need debugging, fixture updates, and locator repairs. AI can speed up creation, but it does not remove maintenance.

That is why I like comparing the first version of a suite with its first maintenance cycle. The real question is not "how fast can we create tests," it is "how expensive is the second week of ownership?" The piece Endtest vs Hand-Written Playwright Suites: What Changes After the First Maintenance Cycle makes that tradeoff concrete, especially around upkeep, collaboration, and debugging.

If your app UI changes often, especially locators, editable regression suites can reduce friction. They let the team maintain tests without rewriting everything every time a selector moves. That is why the guide on How to Use Endtest for Editable Regression Suites When Your Team Keeps Changing Locators is relevant, because it frames locator stability as a maintenance problem, not just a tooling preference.

What to trust in AI testing tools

We should also be careful about the tools themselves. A tool saying it has AI features does not tell us much. Does it explain why it healed a test? Can a human review the change before it lands? Does the generated test code stay understandable six weeks later? Those details matter more than the label.

Before we trust any automation that claims to be smart, we should verify how much control we keep. The checklist in AI Features in Testing Tools: What Buyers Should Verify Before Trusting the Automation is a good reminder to look for explainability, human review, and failure visibility. That is the difference between a helpful assistant and a black box that quietly erodes confidence.

The hidden cost of generated tests

Generated test code is not free just because the first draft appeared quickly. Someone still has to review it, debug it, align it with the app architecture, and keep it from turning into a pile of near-duplicates. If the team does not budget for that work, the automation suite becomes harder to trust over time.

This is where AI-assisted development can mislead teams. A fast start can hide a slow tail. The article The Hidden Cost of AI-Generated Test Code is a useful counterweight, because it frames review, infrastructure, and long-term maintenance as part of the real cost of ownership.

A practical operating model for the team

Here is the working approach I would use.

1. Let AI draft, but not decide

Use AI to produce a first pass for test ideas, boilerplate, and edge case lists. Do not let it decide what matters. A human should pick the assertions, the test boundaries, and the priority of the suite.

2. Review for behavior, not just syntax

When reviewing AI-assisted code, ask three questions, does this test protect a user outcome, does it fail for the right reason, and is the setup readable enough for a teammate to fix later?

3. Keep regression suites editable

If selectors, flows, or copy change often, prioritize maintainable regression patterns over raw code volume. The suite should be easy to update without a full rewrite.

4. Test AI features with properties and evals

For chat, summarization, classification, and similar features, define what good output means. Use assertions and curated eval sets rather than manually reading every response.

5. Measure ownership, not just generation speed

When comparing tools or approaches, include the cost of the first maintenance cycle. That is where the real shape of the workflow appears.

Where this leaves us

AI-assisted development is changing testing, but not in the dramatic way tool vendors like to suggest. It does not eliminate QA work. It changes where the work happens. We spend less time typing repetitive code and more time checking assumptions, keeping suites maintainable, and deciding which failures actually matter.

If we get this right, AI can help the team move faster without turning testing into guesswork. If we get it wrong, we end up with more code, more tests, and less confidence.

For teams still evaluating tools and workflows, Best AI Testing Tools for QA Teams is a practical overview of no-code, low-code, and code-first options. Use it as a starting point, then judge every option by the same rule, does it reduce maintenance without hiding risk?

That is the bar I would set for the next quarter, better coverage, clearer review, and automation that stays usable after the first rush of AI-generated output.

A Practical Note on Testing in Release Pipelines Without Slowing the Team Down

Antoine Dubois — Wed, 03 Jun 2026 19:11:09 +0000

Team, we need to tighten release quality without turning the pipeline into a traffic jam.

The goal is not more tests for the sake of more tests. The goal is a release path that tells us, quickly and reliably, whether we can ship. That means testing has to fit the pipeline, not sit beside it as a separate ritual that gets skipped when people are busy.

What testing should do inside CI/CD

Testing in a release pipeline has three jobs.

First, it should catch obvious breakage early, close to the commit that caused it. Second, it should protect the release from known risk areas, especially the paths we do not want to debug at 5 p.m. on a Friday. Third, it should give release owners enough signal to make a decision without opening six dashboards and asking three different teams what happened.

If tests do not support one of those jobs, they are probably in the wrong place, too expensive to run, or too flaky to trust.

The practical split is usually simple:

fast checks on every pull request
broader integration checks before merge or before release
a small set of release gate tests that are stable enough to mean something
targeted post-deploy verification for production risk

That sounds basic, but teams still get tripped up because the pipeline grows without a clear contract. A test suite starts as a safety net, then becomes a junk drawer.

Make release gates narrow and meaningful

A release gate should answer one question, can we ship this version with acceptable risk?

That means your gate should focus on the handful of flows that would hurt most if broken. Login, checkout, payment, permissions, data writes, feature-flagged behavior, whatever is most critical in your system. You do not need every scenario at the gate. You need the right scenarios.

This is where teams often confuse smoke testing with sanity testing. Smoke checks that the build is not obviously dead, sanity checks that a specific change or area still makes sense after a small update. The distinction matters because it keeps the pipeline honest about what each test stage is for. The article Smoke Testing vs Sanity Testing: What’s the Real Difference? is a useful reminder that not every test should carry the same release weight.

A good rule is to make gate tests boring. If a gate test fails, it should usually mean one of two things, the product is broken or the environment is broken. If the answer is often "maybe the test was flaky," then the gate is not doing its job.

Flaky failures are a release management problem, not just a test problem

Flaky tests do more damage than a few missed bugs. They teach teams to ignore red builds, rerun until green, and treat signal like noise. Once that habit sets in, the pipeline loses credibility.

The fix is not just "retry more". Retries can be part of the strategy, but only after you understand the failure pattern. If a test fails due to timing, isolation, data setup, network dependency, or environmental drift, you need to attack the real cause.

For GitHub Actions specifically, there is a good practical guide in How to Stabilize Flaky E2E Tests in GitHub Actions. What I like about this kind of guidance is that it treats flakiness as an engineering workflow issue, logs, artifacts, environment parity, and clearer debugging, not just as a test authoring mistake.

A few reliability habits pay off quickly:

isolate test data so runs do not collide
keep environment setup consistent across local, CI, and staging
capture screenshots, logs, and network traces on failure
quarantine flaky tests fast, but require a fix path
track rerun rates, not just pass rates

That last point matters. A suite that passes after three retries is not stable. It is expensive optimism.

Environment control and test data are part of the product

A lot of CI pain comes from pretending test environments are interchangeable.

They are not.

If a release pipeline depends on shared environments, mutable test data, or manual resets, then reliability will always be weaker than the code quality deserves. Fast teams need strong environment control, predictable data seeding, and clear ownership when something in the test environment drifts.

That is why vendor selection and partner evaluation should include the unglamorous stuff. The article How to Evaluate a QA Outsourcing Partner for Test Data, Environment Control, and Release Coverage is a good example of what to look for. Even if you are not outsourcing QA, the criteria are still useful internally, because they describe the real operational concerns, test data handling, environment control, release coverage, escalation paths, and reporting quality.

If your team cannot answer these questions quickly, you have a process problem:

Questions worth asking

Who owns the test environment when it breaks?
How is test data seeded, refreshed, and cleaned up?
Can the same scenario be reproduced in staging and CI?
What is the escalation path when release coverage misses something important?

You do not need perfect environments. You need environments that are controlled enough to trust and fast enough to maintain.

Release coverage should reflect risk, not habit

Coverage is not the same as confidence.

A lot of teams keep old regression packs alive because "we have always run them." That is how pipelines get slower without getting safer. A better approach is to map test coverage to release cadence and failure cost.

If a service changes every day, the regression strategy should be lean, automated, and selective. If a release touches payments or customer data, coverage should be stronger around those flows. If a change is behind a feature flag, validation should include both the enabled and disabled states, plus the fallback behavior.

The article How to Evaluate an Outsourced Regression Testing Partner for Release Cadence, Coverage, and Escalation Speed makes this point well, because it focuses on cadence and triage speed instead of just raw test count. That is the right mindset for internal teams too.

A release pipeline should not ask, "Did we run the big suite?" It should ask, "Did we cover the risks that changed?"

Feature flags reduce risk, but they also add test surface

Feature flags are useful because they let teams ship code separately from exposure. But flags do not remove testing work, they change it.

Now you need to validate combinations, flag states, user targeting, fallback behavior, and gradual rollout. If you do not, you can create a new class of release bug where the code works, the flag works, and the rollout still fails.

A practical breakdown is to test the default-off path, the default-on path, the targeted-on path, and the rollback path. You also want to know what happens when a flag service is slow or unavailable.

For a deeper walkthrough, How to Test Feature Flag Rollouts Without Creating a New Class of Release Bugs is a solid reference. It lines up with how teams actually ship now, where the release problem is often not "does the code compile," but "what happens when we expose this to 5 percent of users first?"

Reporting should help release managers make a decision

If your test reporting only helps the engineer who wrote the test, it is incomplete.

Release managers, QA leads, and execs all need different levels of detail, but they need the same basic truth, what failed, how often, how risky it is, and whether the failure blocks release. A good report should let someone drill from summary to defect to evidence without reading raw logs unless they want to.

That is why reporting tools should be evaluated with the release decision in mind. The article How to Evaluate a Test Reporting Tool for Release Managers, QA Leads, and Executives is useful because it frames reporting around dashboards, defect trends, traceability, and stakeholder-friendly summaries. That is the shape of reporting teams actually need.

A solid release report answers:

what changed since the last run
what failed, and whether it is new or known
which tests are flaky versus genuinely broken
whether the failure blocks deployment
who owns the next action

If a report cannot answer those questions in under a minute, it is too noisy for a fast team.

A lightweight operating model for fast teams

If I had to keep this simple, I would use this model:

1. Keep the fast lane fast

PR checks should stay short, deterministic, and easy to read. They are there to catch local mistakes before they become shared mistakes.

2. Keep the gate small

Only the most release-critical flows should block shipping. Everything else can be covered earlier, later, or through targeted checks.

3. Treat flakes like incidents

A flaky test is not just annoying, it is a reliability issue. Give it ownership, severity, and a fix deadline.

4. Control the environment

Stable pipelines need stable data and stable infrastructure. If either one is drifting, test confidence will drift too.

5. Make reporting decision-ready

The output of testing should help someone say yes, no, or not yet.

Next steps

If your release process is already moving fast but quality feels fragile, do not start by adding more end-to-end tests. Start by asking where the pipeline lies to you.

Look for the places where red builds are ignored, where reruns are common, where environments are inconsistent, and where reports create more questions than answers. Then tighten the system around those weak points.

The best release pipelines are not the ones with the most automation, they are the ones that can be trusted when the team is under pressure.

A Curated List of Articles About Modern Software Testing

Antoine Dubois — Tue, 02 Jun 2026 21:49:45 +0000

Software testing is changing quickly. Teams are dealing with faster release cycles, more AI-assisted development, more complex browser behavior, and higher expectations around product quality.

I collected a few practical articles that cover different parts of modern QA, test automation, developer workflows, and testing strategy.

Final thoughts

The useful thing about these topics is that they are connected. Tool selection, browser coverage, AI-assisted workflows, CI reliability, maintainability, and team adoption all affect whether test automation actually works in practice.

Hopefully these resources help you compare options more clearly and avoid some of the common traps teams run into when scaling QA automation.

How AI-Assisted Development Changes the Way We Test, Review, and Automate

Antoine Dubois — Mon, 01 Jun 2026 15:40:09 +0000

AI-assisted development is changing more than how fast teams ship code. It is changing the shape of risk.

When a developer can generate a component, a test, or even a small feature in minutes, the old assumptions around review and coverage start to break down. The question is no longer just, "Did we write enough tests?" It becomes, "Do we understand what the code is doing, what the test is actually proving, and where the system may be guessing?"

That shift matters for QA engineers, developers, and platform teams alike. AI can be a useful multiplier, but it can also create a false sense of confidence. The teams that do well with AI-assisted development are usually not the ones that automate the most. They are the ones that make test intent, traceability, and reliability more explicit.

AI changes the meaning of coverage

Traditional coverage conversations often focused on lines, branches, and a rough sense of feature completion. Those numbers still have value, but they do not tell the whole story when code is AI-generated or AI-assisted.

A code assistant can produce a lot of syntactically valid output very quickly. That means you can end up with a larger surface area of code, but not necessarily a larger surface area of confidence. In practice, I find it more useful to ask three questions:

1. What user behavior is covered?

A test suite should protect business-critical paths, not just code paths. If AI helps generate a new form, a new endpoint, or a new workflow, the important question is whether the tests still describe the user journey correctly.

2. What assumptions are hidden?

AI-generated code often contains implicit assumptions about defaults, timing, retries, API responses, or DOM structure. Those assumptions are easy to miss in review. Good testing makes them visible.

3. What changed in the failure mode?

When the system fails, is it failing because the logic is wrong, the selector is fragile, the environment is unstable, or the model output is inconsistent? Different failure modes need different strategies.

That last point is one reason I like the observability mindset from AI Test Observability Checklist: Metrics That Reveal When Your Agent Is Guessing. Even if your team is not testing an AI agent directly, the article is useful because it pushes you to look for signs that automated behavior is drifting, guessing, or becoming flaky. That is a strong mental model for AI-assisted QA too, especially when test steps or assertions are generated with help from a model.

Review needs to become more explicit, not just faster

One of the easiest traps with AI-assisted development is assuming that speed automatically improves review throughput. In reality, faster code creation can make review harder, because reviewers see more changes and have less time to understand the intent behind them.

That means code review and test review should become more explicit about purpose. I like to see reviews answer questions such as:

What risk does this change introduce?
What evidence proves it works?
What would make this test give us a false pass?
If this fails in CI, how would we know whether it is a product defect or a test defect?

This is especially important for generated tests. A test that passes is not always a good test. It might be asserting too little, depending on timing too much, or validating a DOM detail that has no user value.

A useful supporting perspective comes from How to Measure Browser Test Stability Without Confusing Real Failures With Flakes. The key lesson is simple, stability metrics are only useful when they separate genuine regressions from noisy behavior. In an AI-assisted workflow, that separation matters even more because you may be creating more automation faster, which also increases the chance of introducing brittle checks.

Coverage is now about trust boundaries

When AI helps produce code, I think about coverage in terms of trust boundaries.

A trust boundary is the point where your team stops assuming something will behave correctly and starts verifying it. In AI-assisted development, those boundaries often include:

generated UI flows that rely on specific selectors
API calls that depend on loosely specified response shapes
transformations where the code compiles, but the business logic is easy to misread
tests that were authored quickly and never reviewed for negative paths

The practical move is to increase coverage where the cost of failure is high, and reduce coverage where the value is low. That sounds obvious, but AI makes it easier to overproduce low-value tests. You can end up with a large suite that looks impressive and still misses the risky paths.

This is where roadmap thinking helps. A good automation plan should not just ask what can be automated next, it should ask what should be automated next, based on risk and maintenance cost. The guide How to Build a QA Automation Roadmap for the Next 12 Months is a solid reminder that sustainable automation depends on prioritization, team capacity, and ROI, not just enthusiasm.

Automation strategy changes when code generation is cheaper

AI-assisted development changes the economics of automation. If code is easier to generate, then creating a test suite is easier too. But maintaining it is still the hard part.

That changes the tool conversation. Low-code platforms, code-based frameworks, and hybrid approaches all look different when the team can move faster on implementation. What matters most is not whether a tool is flashy, but whether it supports change without turning every edit into a mini-migration.

That is why I think comparisons like Endtest vs Low-Code Test Automation Platforms: What Changes in Maintenance, Collaboration, and Scale are relevant to AI-assisted teams. The article is useful because it frames automation around maintenance, collaboration, and scaling, which are exactly the pain points that grow when more tests are created with AI help.

For teams adopting AI-assisted coding, the main automation question is not, "Can the model generate this test?" It is, "Can the team understand, update, and trust this test six months from now?"

Debugging still needs human discipline

AI can help draft a test or suggest a fix, but it does not replace good debugging habits. In fact, AI can make sloppy debugging easier, because it offers plausible explanations very quickly.

That is dangerous. If a test fails only in WebKit, or only under CI load, or only after a certain navigation path, the worst thing you can do is accept the first explanation that sounds reasonable.

I keep coming back to workflows like How to Debug WebKit-Only Failures in Playwright Without Guessing. The important part is not the browser engine itself, it is the discipline: use traces, compare logs, inspect timing, and validate assumptions against a real browser. That approach is even more important when the test or the component was generated with AI help, because you need a reliable way to distinguish a real compatibility issue from a bad test design.

What this means for outsourced teams and handoffs

AI-assisted development also affects the way work gets handed off between developers, QA, and external partners. If a team expects QA to verify generated features without clear test intent, handoffs become vague very quickly.

The best handoffs are specific. They describe what changed, why it matters, what should be observed, and which parts of the suite are supposed to protect the change. This is true whether QA is internal or outsourced, but it matters even more when AI has accelerated the pace of delivery.

I found the discussion in Endtest vs Selenium for Outsourced QA Teams: What Changes in Maintenance, Handoffs, and Time to Value helpful here, because it focuses on maintenance burden, handoff quality, and time to value. Those are exactly the places where AI-assisted delivery can either reduce friction or amplify confusion.

If code changes arrive faster, QA needs better context, not just more tickets.

A practical mindset for AI-assisted QA

If I had to reduce all of this to a few habits, they would be these:

Make test intent visible

Every automated test should answer a clear question. If you cannot describe the behavior it protects, the test is probably too vague.

Review generated code as if it were unfamiliar code

Even if the assistant produced it, a human still owns the result. Read it for assumptions, edge cases, and maintainability.

Treat flakiness as a product signal

Flaky tests are not just a CI annoyance. They often reveal unstable selectors, weak assertions, or unclear ownership.

Prefer stable evidence over clever automation

A simple, well-structured test with good diagnostics is better than a complex one that is hard to trust.

Revisit the suite as the team’s AI usage grows

The more AI contributes to implementation, the more important it becomes to audit test quality, failure signals, and maintenance cost.

Closing thoughts

AI-assisted development is not making testing less important. It is making the quality of testing more visible.

Teams that rely on AI to write code, tests, or even review suggestions still need the same fundamentals: clear intent, stable assertions, good observability, and a realistic maintenance plan. What changes is the speed and scale at which weak practices show up.

If anything, AI raises the bar. It rewards teams that can explain why a test exists, what it protects, and how they know it is telling the truth. That is a good thing. It pushes QA back toward its real job, not just confirming that code runs, but proving that the system behaves in ways the team can trust.

Can AI help us automate tests faster?

Antoine Dubois — Fri, 15 May 2026 07:46:10 +0000

Just like most of you, I'm writing more code than ever in the last year.

I feel like I cloned myself 5 times.

But my blood tension rises when I see Claude saying "You're absolutely right!" after it inserted a bug.

And we did introduce a significant number of bugs in production in the last few months.

I'm old enough to remember the days when testing was taken more seriously, I don't know what happened after that.

Maybe some younger devs assumed that cross-browser issues are no longer a thing? Because they sure are.

I'm looking at a list of AI Testing Tools and trying a few of them.

The last thing I want is to write Selenium or Playwright code and having to maintain a separate codebase just for the tests.

I want some fancy AI tool that creates tests and then takes care of them and just informs me if the tests are passing or not.

Seems reasonable, right?

Staying Healthy as a Dev Who Works 10 Hours a Day

Antoine Dubois — Thu, 14 May 2026 22:11:20 +0000

We devs spend most of the day sitting, thinking, debugging, building, shipping, and then somehow “relaxing” by sitting again in front of Netflix (ew).
After so many hours of work, the easiest thing is to crash on the couch, order something greasy, and tell yourself you’ll fix your life on Monday.

But health doesn’t really collapse in one dramatic instance. It slowly gets eaten away.

One skipped workout.

One evening where you don’t move.

One “I deserve this” meal every night.

One weekend spent indoors.

And then years pass and we end up with regrets.

For me, the goal is not to become a fitness influencer. The goal is to still feel like a functional human being after years of building software.

My basic routine:

I try to go to the gym 3 times per week. Nothing crazy. Just enough strength training to remind myself that I’m not just a brain attached to a keyboard.

We have replaced Netflix evenings with long walks. Walking is underrated. It clears your head, gives your eyes a break, and helps you process problems without staring at another screen.

On weekends, I like running in the park. Again, not because I’m training for some marathon. It just feels good to move outside, breathe fresh air, and remember that the world is bigger than tickets, commits and emails.

Food is another part of it. I don’t eat everything I crave. That doesn’t mean living like a monk.

It just means not treating every stressful day as a valid reason to eat whatever I want.

The older I get, the more I think about the 80-year-old version of myself.

I don’t want to be retired and broken.

I want to be the kind of 80-year-old who learns guitar, goes for walks, does volunteer work, reads books, tells stories, and still has enough energy to be curious about life.

That version of me is not built at 80.

He is built now.

During the busy years.

During the 10-hour workdays.

During the evenings when Netflix is easier than walking.

During the moments when I either go to the gym or invent a very convincing excuse.

Being a dev can be mentally intense, but it should not slowly destroy our bodies.

You don’t need a perfect routine.

You just need one that keeps future you alive, mobile, and still interested in the world.

What's your routine for staying healthy?