A few years ago, creating the automated test was usually the difficult part.
You had to choose a framework, learn its APIs, build a project structure, write selectors, configure a runner, and eventually persuade the test to pass on a developer machine.
That work still exists, but AI has made the first draft dramatically easier. A developer can describe a flow and get Playwright, Selenium, or Cypress code in seconds. A no-code platform can record a journey or generate editable steps from plain English.
The bottleneck has moved.
The difficult part now is building a test automation system that still produces useful information six months later, after the UI has changed, the team has grown, the CI environment has become busier, and the person who created the original tests is working on something else.
That changes how I evaluate test automation.
I care less about how quickly a tool can produce a successful demo and more about what happens after the demo.
Modern web applications are not clean, predictable DOM trees
Many test automation examples still use pages where every button is visible in the main document and every element has a stable identifier.
Real applications are messier.
A checkout flow might contain a payment form inside a cross-origin iframe. A design system might use nested web components and Shadow DOM. Authentication, chat, analytics, consent, and support widgets may all be injected by third parties on their own schedules.
This is why I think a serious evaluation should start with the hardest surface in the application, not the easiest login form.
A useful checklist is covered in How to Evaluate a Test Automation Platform for Shadow DOM, Iframes, and Embedded Third-Party Widgets.
The interesting question is not simply whether a platform claims to support iframes or Shadow DOM. It is whether the entire workflow remains understandable:
- Can the test enter and leave browser contexts reliably?
- Can it locate controls without depending on fragile implementation details?
- Does it explain which document or component it searched?
- What evidence is captured when a vendor widget loads differently in CI?
- Can the team repair the test without reverse-engineering generated selectors?
Support shown in a feature table is not the same as support that survives production use.
A failure without evidence is just another task for a human
AI test agents are often described as systems that can run a test, notice a problem, retry the step, and sometimes recover automatically.
That can be useful, but recovery is not the same as understanding.
Imagine that an agent retries a failed flow and the second execution passes. Was the original problem caused by the application, the test, the browser environment, the data, or a different decision made during replay?
Without enough evidence, “the retry passed” is not a diagnosis.
The observability model described in What to Log When an AI Test Agent Replays a Failed Run and Still Can’t Explain the Failure is a good way to think about this.
A useful failed-run package should connect the test steps with the surrounding state:
- the exact application version and environment;
- feature flags and user permissions;
- the locator selected by the test;
- fallback locators or recovery actions;
- screenshots, DOM state, network activity, and console errors;
- test data created or modified during the run;
- the point where a replay diverged from the original execution.
This is especially important when AI is making decisions.
I do not need a giant wall of synthetic reasoning. I need a concise decision trail: what the agent expected, what it observed, what alternative it chose, and how confident it was.
Good automation does not merely say that something failed. It helps the team decide what to do next.
Framework comparisons need a real workflow
“Which framework is best?” is usually too broad to produce a useful answer.
The answer changes when the workflow includes dynamic tables, inline editing, search filters, animated navigation, API setup, multiple browsers, or frequent UI redesigns.
For example, Endtest vs Cypress for Testing Dynamic Tables, Search Filters, and Inline Row Actions looks at a much more specific problem than a generic feature comparison.
That specificity matters.
A dynamic table test may need to:
- Create a record with known data.
- Wait for an asynchronous refresh.
- filter or sort the table.
- find the correct row without relying on its position.
- trigger an action inside that row.
- verify both the visible result and the backend state.
Cypress can give a code-first team detailed control over that workflow. Endtest can be more attractive when QA needs to create and maintain the coverage without owning another framework codebase.
Neither conclusion can be reached by comparing only syntax.
The same is true in broader framework discussions. Selenium vs Cypress in the AI Era is interesting because AI changes the cost of producing code, but it does not remove the ownership model behind that code.
AI can generate page objects, helpers, fixtures, and assertions. The team still owns the architecture, reviews, dependencies, failures, browser infrastructure, and future rewrites.
Generated code is cheaper to create. It is not automatically cheaper to operate.
Use APIs for state and the browser for behavior
One of the best ways to improve browser tests is to stop using the browser for every part of the setup.
Suppose the behavior under test is whether an archived project appears correctly in a dashboard.
Creating a user, logging in, navigating through several screens, creating a project, editing it, and archiving it through the UI may add several minutes and many possible failure points before the test reaches the thing it actually needs to validate.
An API call can create the required state directly. The browser can then verify the user experience.
How to Combine API Calls with Playwright Tests explains this pattern well: use direct requests for setup, cleanup, authentication, and backend verification, while keeping browser steps focused on behavior only a browser can prove.
I use a simple rule:
Use the UI when the interaction matters. Use an API when the state matters.
There are exceptions. A signup test should probably use the signup interface. A form-validation test should not bypass the form.
But most large suites contain many browser steps that exist only because the team never created a cleaner data strategy.
Removing those steps often improves speed, stability, and clarity at the same time.
Test the release mechanism, not only the final page
Modern releases are rarely a single switch from old to new.
Features may be enabled for employees first, then a small customer segment, then a percentage rollout. Different users can see different behavior at the same time. A revert may disable the visible feature while leaving data or background jobs behind.
That means the test plan needs to cover the release mechanism itself.
How to Build a Test Plan for Feature Flags, Rollouts, and Safe Reverts in Web Apps provides a useful model for testing both sides of a flag, rollout targeting, transitions between variants, and rollback behavior.
A feature-flag test plan should answer questions such as:
- Does the old experience still work when the flag is disabled?
- Does the new experience work for every intended role and account type?
- What happens when the flag changes during an active session?
- Are API contracts compatible with both versions?
- Can the release be reverted without corrupting new data?
- Do analytics and audit events identify which variant was active?
The visual transition between states matters too.
Animated route changes and CSS View Transitions can create failures that look like locator problems even though the element is technically present. A control may be moving, covered, duplicated temporarily, or attached to a document that is being replaced.
How to Reduce Flaky Browser Tests Caused by Animated Route Changes and CSS View Transitions is a useful reminder that waiting for an element to exist is not always the same as waiting for the interface to become interactive.
The correct synchronization point is usually a meaningful application state, not an arbitrary delay.
CI integration is part of the test design
A test that works locally but cannot provide a reliable result to the delivery pipeline is not finished.
CI needs more than a command that starts a browser run. It needs to know:
- whether all expected tests actually finished;
- which build and environment were tested;
- whether a failure was functional or infrastructural;
- where the relevant artifacts are stored;
- whether deployment should continue.
How to Integrate Test Automation with CircleCI describes the part that is often overlooked: turning remote test execution into an explicit release gate.
The pipeline should wait for completion, retrieve an aggregate result, print a useful summary, preserve links to evidence, and return a failing exit status when the release should be blocked.
Triggering a test is easy.
Teaching the delivery system how to trust the result is the real integration.
Browser infrastructure has an ownership cost
Eventually, a browser suite needs somewhere to run.
At that point, teams often compare a self-hosted Selenium Grid with a browser cloud such as BrowserStack. On paper, the self-hosted option may appear less expensive because the software is open source.
The missing line item is ownership.
Selenium Grid vs BrowserStack frames the choice correctly: which failure modes does the team want to own?
With a self-hosted grid, the team may own browser images, drivers, node health, scaling, routing, video capture, security, upgrades, and capacity planning.
With a browser cloud, the subscription is more visible, but much of that infrastructure work moves to the vendor.
This does not mean that a managed cloud fixes poor tests. Fragile selectors and shared test data remain fragile anywhere.
It does mean that infrastructure should be included in the total cost of the automation strategy. The framework license is usually a tiny part of that calculation.
Replacing manual regression is an operating-model decision
Teams sometimes treat the move from manual regression to automation as a writing project:
We have 300 manual cases. Let’s convert them into 300 automated cases.
That approach tends to reproduce the old process in a more expensive form.
A better question is which risks need repeatable evidence on every release, which scenarios need broad browser coverage, and who can realistically maintain the resulting system.
The article Endtest Buyer Guide for Teams Replacing Manual Regression on Fast-Changing Frontends makes an important point: the ongoing maintenance tax matters more than the first week of automation.
This is where platforms such as Endtest become interesting.
A code-first framework can be the right choice for teams that want maximum control and already have people committed to maintaining the framework. But a platform with editable tests, integrated execution, self-healing, API steps, cross-browser infrastructure, and understandable run evidence can create a different ownership model.
QA can own more of the coverage directly. Engineers can still participate without becoming the only people capable of changing a test. The organization buys a working testing system rather than assembling every layer separately.
That broader view is also reflected in Best AI Test Automation Tools in 2026: My Practical Shortlist, which places Endtest first and evaluates tools on their ability to create, run, debug, and maintain complete end-to-end workflows rather than on AI generation alone.
That is the evaluation standard I find most useful.
The tool should reduce the amount of testing infrastructure your team has to think about
There will always be teams that genuinely benefit from building their own framework.
They may have unusual technical requirements, mature platform engineering, strict source-code ownership rules, or custom environments that cannot be handled well by a general-purpose product.
But many teams are not trying to invent a testing platform. They are trying to release a web application with fewer regressions.
For those teams, the best solution may be the one that removes the largest number of secondary responsibilities:
- selector repair;
- browser provisioning;
- result storage;
- video and screenshot capture;
- permission management;
- test data utilities;
- API, email, SMS, file, and database validation;
- parallel execution;
- CI result handling;
- long-term framework maintenance.
The open-source library may be free, but the complete operating system around it is not.
For a more framework-focused discussion, the video Playwright vs Selenium compares the two beyond the usual speed benchmarks.
For teams considering a higher-level platform instead, Best No-Code Test Automation Tools in 2026 reviews Endtest, Mabl, testRigor, Ghost Inspector, Reflect, and TestMu AI from a practical usability perspective.
Final thought
The first successful automated test is no longer a meaningful milestone by itself.
The real milestone is when the team can run the suite repeatedly, understand failures quickly, trust the release signal, and update coverage without starting a small internal tooling project every time the frontend changes.
That is why the most important test automation questions in 2026 are operational:
- Who will maintain this?
- What evidence will a failure produce?
- How will it behave on complex browser surfaces?
- How will data be created and cleaned up?
- How will CI interpret the result?
- Who owns the browser infrastructure?
- Can the suite survive normal product change?
AI can accelerate test creation.
The winning tools will be the ones that reduce the work surrounding the test.
Top comments (0)