DEV Community: Mark Noonan

Skip tests that pass on the current build: Re-run optimization in Cypress Cloud

Mark Noonan — Thu, 01 Oct 2026 16:35:34 +0000

When Cypress detects a CI rerun, it can now skip the work that already passed and run only what failed, so you get to green faster, burn fewer CI minutes, and record fewer billable test results. Re-runs with few failures can skip 99% or more of the tests that would have re-executed. It works out-of-the-box with common CI providers, and ships with overrides to customize the re-run behavior to your specific needs.

Re-run Optimization is available for all Business plans and higher, and in free trials, as a part of Smart Orchestration.

This post will walk through the highlights for saving time and getting a green build faster.

Why do teams re-execute failed Cypress runs?

We interviewed current Cypress Cloud customers to get this feature right. Most told us they already use tools like flake detection to stay on top of things they can control from the testing side. Generally their test design is not causing flakiness, and in-run, often flake represents real race-condition bugs in the application logic.

Re-runs address instability that’s not at the application layer or the test design layer. A re-run adds information to help you understand the full end-to-end picture, especially if your testing environment itself is not fully stable, which means a single failure could have multiple causes. The re-run helps rule some of those causes out cheaply, before you or your AI systems work on debugging.

For busy teams with large test suites on complicated applications, keeping the test environment stable isn’t always practical, or even necessarily desired. When systems integrate with live devices or depend on third parties, stability itself is part of what a full end-to-end test run is measuring.

Whatever the underlying reason for a failure, if something is blocking a release or an important merge, your priority in that moment is to understand the issue and get unblocked as fast as possible. At those times, re-run optimization makes your re-executions cheaper, faster, and less painful.

How much time is saved with Re-run optimization in Cypress Cloud?

The most time savings come from test-level re-runs, which allows Cypress to cherry-pick individual tests that need to re-run. Based on real production runs from customers who have adopted this technique, the median re-run skips 92% of the original run’s tests, and finishes 9.3 minutes faster than the original run.

The maximum savings we have seen on re-runs are when there are large test suites with small numbers of failures, where 99.8% of the tests that would have re-executed were skipped , and 36 minutes were saved for the test run. This is the pattern you may see if you run long nightly test suites or pre-release tests of live environments that might have deployments or other disruptions during the test execution.

Re-run optimization makes it much faster and more practical to get that extra confidence and confirmation of what’s going on when a build has failed, and avoids the risk of instability leading to the original failures passing on the new run, only for something else to start failing.

Choose spec-level or test-level reruns in Cypress Cloud

You set the granularity once in the Smart Orchestration section of your organization or project settings, and projects can inherit the org default or override it.

When set to run only failed specs , Cypress re-runs every spec that contained a failure on the last run, including the tests in that spec that passed, which is the safer choice if your tests share state inside a spec.

When set to run only failed tests , Cypress re-runs only the tests that didn't pass, plus any that are new or changed since the last run. It requires Cypress 15.21.0 or later (older versions fall back to failed specs). If a test would pass on its own with .only, it will work in a re-run. If not, prefer spec-based retries for your project.

Controlling the re-run behavior in Cypress Cloud

If you re-run before Cloud marks the previous run complete, Cypress plays it safe and runs everything. Keeping your run completion delay short can reduce the window where your CI is ready to rerun a job but Cypress Cloud is still open. But in many cases, you may want to manage the previous run’s status explicitly. Our recently released Run Cancellation API, and Run Completion API give you programmatic control over the run state. Combined with Spec Prioritization and Auto Cancellation.

Turn it on

Re-run optimization is part of Smart Orchestration on Business and Enterprise plans, and it's available during a free trial. Owners and Admins can enable it in organization or project settings. The Re-run optimization docs cover setup for each CI provider.

Build custom workflows with Cypress Cloud Webhooks

Mark Noonan — Tue, 08 Sep 2026 17:17:31 +0000

Webhooks are a new addition to Cypress Cloud, supporting more flexible automation workflows beyond what is built into the platform itself.

Learn why we've added them and how to get started using them in your project.

Integration support in Cypress Cloud

Cypress Cloud supports a wide range of integrations out of the box. If you need a GitHub status check for every group of tests in your test run, we've got you covered. Slack and Teams messages when a run fails? That's there, along with desktop notifications, Jira ticket creation, high-level trend reporting APIs, and more.

Why webhooks?

Using webhooks is a great solution for custom integrations where you want Cypress Cloud to push run results and other data into another system.

A webhook can be the starting point to kick off workflows based on inspecting the outcome of a run, or a way to customize the details you will include in a Slack or Teams message, or send messages to destinations Cypress doesn't support, such as Google Chat.

Webhooks also help you leverage our other recently-released Cypress App and Cypress Cloud updates in ways that work for you. Rather than invoke the Cypress Cloud CLI or MCP at the end of every run, you can set up a URL for Cypress to deliver run results, along with the tags, branch, and commit info that helps you decided if further action is needed.

This lets you be more efficient with AI token usage and reduce rate-limited request you make for routine run analysis. Filter to just the runs you are most interested in based on the webhook payload, then you can dive deeper with scripts using the Cloud AI tools to get Test Replay snapshot data and command logs, or hand off to your LLM using the live Cypress App running locally with cypress tap.

When to use a webhook

In some cases, existing APIs may fully meet your needs, particularly if your runs are small and the existing integrations are well-suited to the content you already want in your messaging platforms.

In other cases, webhooks remove multiple layers of handoffs from your CI environment to other services, removing glue code or even opening up new possibilities, like optionally triggering re-runs of your failed tests in CI based on the tags or commit message you send to Cypress Cloud.

Get started

Available on all paid Cypress Cloud plans, today webhooks can be attached to a run.completed event, which fires when a run has resolved to a finalized state after all results have been recorded.

Admins and owners can add webhooks in your project settings page.

Note: the recently-released Run Completion API may be useful if you have parallel or grouped runs, and you want your webhooks to fire as soon as all expected groups have finished recording.

What's next

We'd love to know how you are using this and what types of events or payloads would be useful for your specific goals. We will continue developing and expanding in this area based on your feedback.

Read the webhooks docs to get your first webhook configured. Don't have a Cypress Cloud account yet? Start your free trial and try it on your own project.

Open Source Accessibility Plugins in Cypress

Mark Noonan — Wed, 19 Feb 2025 16:02:45 +0000

Last year saw the GA launch of Cypress's commercial accessibility testing solution built into Cypress Cloud. We're delighted to see this helping teams speed up and scale their accessibility efforts.

But accessibility testing itself is nothing new in Cypress, and there are many different ways to account for accessibility in test automation which all complement each other. We’ve blogged in the past about considering accessibility in component tests, and recently added an accessibility testing guide to the open-source Cypress App docs.

When drafting that guide I found I wanted to go a bit deeper into the community plugin side of accessibility and Cypress, as I realized that in 2024 alone, there were multiple accessibility related plugins released, some directly related to testing, and others more geared towards customization of Cypress itself.

In this post we’ll take a look at three recently-released accessibility-related Cypress Plugins: Wick A11y, Cypress Runner Themes, and Cypress Voice Plugin, and hear from their creators.

Cypress Voice Plugin

This plugin was created by Dennis Bergiven, who told me "The voice plugin was inspired by your chat with Filip Hric, where you chatted over various ideas and topics using Cypress". This was a really fun live stream Filip and I did back in 2023, and here's the relevant clip called "How to make Cypress talk". In it, I show how to use built-in browser Speech Synthesis to have Cypress verbally announce something at the end of a spec's execution, especially where the spec might contain many tests or do something that takes some time and you have already moved your screen back to the next code change. (In addition to being a great testing educator, Filip is also the developer of the popular Cypress API testing plugin.)

This was just a proof of concept, as we discussed various ways to hack different functionality into Cypress to customize it for specific testing needs. Dennis described later seeing a practical problem at work that this could help with, saying "The idea came back when some colleagues at work expressed they wanted a faster way to detect a test result when running in the UI."

The end result is a highly configurable voice announcer:

Dennis added that "using native web speech API within the plugin fits right into the Cypress framework as it lives in the browser as well", which made the implementation possible.

Cypress Runner Themes

David Ingraham recognized two needs in the community and created a plugin to solve both. First he created a more color-blind friendly theme for the Cypress command in the Cypress App, to avoid using red and green to hint at the passing/failing status of tests. Then he rolled this up into a general Cypress theming package to combine and update some existing theming solutions like cypress-light-theme and cypress-dark.

David said, "I created cypress-runner-themes after a user on the discord mentioned they were having trouble reading Cypress results due to being colorblind and I noticed there wasn't any existing plugins out there. In combination with the old plugin cypress-dark being unsupported, I thought it would be a good idea to provide a newer theme plugin with accessibility support."

Cypress does not have built-in theme support, but David has been able to leverage the fact that Cypress executes plugin code in the browser, and runs in the same environment, to make modifications to the Cypress interface. Plugins are a common way that the community can get out ahead and iterate on behavior that might be desirable in the Cypress App, without waiting for it to be implemented into the core Cypress library. Here's an example of one of the themes:

"It was my first community plugin so I also wanted to learn something new while making a real impact on the community," David told us. We look forward to seeing more in the future!

Wick-A11y

Created by Sebastian Clavijo Suero, and named for its John Wick-style ability to identify and remove accessibility issues with extreme prejudice, this summer blockbuster adds some visual, reporting, and voice-announcing dimensions to the cypress-axe plugin.

Before we hear from Sebastian, it's worth backing up for a second to explain the connection between all these tools.

At the root is the open source Axe Core library by Deque Systems, which was released in 2015. Axe Core has over 1 billion downloads and is used in many accessibility testing systems such as Google's Lighthouse, as well as in Cypress Accessibility. (Side note: we were delighted to welcome Dylan Barrell, the creator of Axe Core, to Cypress Conf 2023, where he shared his presentation about Agile Accessible Software Development.)

The cypress-axe plugin by Andy Van Slaars was first published in 2018, providing an easy integration in Cypress allowing testers to add Axe Core checks where they chose at different steps of the user journey in their tests. In our docs, we discuss more about these kinds of in-test accessibility scans work, so I won't repeat that here.

While there are other accessibility checkers that also have Cypress integrations, cypress-axe has been the go-to for most Cypress users.

OK, now we're up to date!

Much like the other plugin authors, Sebastian saw an opportunity for extending beyond the current behavior to solve a problem, and shared that solution with others. I send him over a few questions:

What inspired you to create wick-a11y?

I created Wick-a11y to significantly simplify accessibility testing, which is often neglected despite its crucial role in developing inclusive web applications. Although tools like cypress-axe are easy to set up, they provide raw data that requires additional processing for clear interpretation of the results. Wick-a11y seamlessly integrates with Cypress, delivering an easy-to-use solution to assess accessibility, interact with results directly in the Cypress runner, and effortlessly produce detailed HTML reports with screenshots.

What made Cypress a good platform for this?

Cypress is an excellent platform for wick-a11y due to its seamless integration capabilities with tools like axe-core for comprehensive accessibility testing. Cypress's interactive test runner and user-friendly interface allow for real-time interaction with accessibility results directly on the web page, enhancing the identification and resolution of issues. Additionally, Cypress supports detailed logging, providing testers with an efficient way to document and share accessibility findings.

Tell us about your recent addition of voice feedback

I added the voice feedback feature to wick-a11y, acknowledging that many individuals testing website accessibility are affected by disabilities themselves. By providing auditory cues alongside accessibility violations, this feature enhances the testing experience, empowering those committed to inclusivity to engage without barriers and continue championing a more accessible web for everyone.

Here's an example of a test open in the Cypress App, showing some accessibility issues detected with cypress-axe and highlighted by wick-a11y

There is much more to see, so check out the links below to learn more about adding these kinds of accessibility assertions to your Cypress tests.

Links

What comes next

The Cypress community has created a useful array of plugins to support accessibility testing and other customized extensions to the app. This is only a recent subset of plugins related to accessibility in Cypress - there are plugins for other accessibility testing packages beyond Axe Core, as well as accessibility-minded test locators from Cypress Testing Library, and more.

What I like about these three recently released plugins is that they represent extensions and iterations upon existing ideas to solve new needs. They leverage the hackability that's inherent to the browser-based nature of Cypress to great effect, and they set the baseline for future iterations to continue building.

Do you have a plugin for Cypress? You can submit it to our documentation's plugins list.

Why I rarely use `getByRole`: Testing Library and the first rule of ARIA

Mark Noonan — Sat, 04 May 2024 00:16:34 +0000

Note: I've gotten some feedback from a Testing Library maintainer, Matan Borenkraout, about these ideas. I haven't gotten around to incorporating that feedback yet. He had some good examples where ByText or even ByLabelText might not be able to find some elements due to how the label is created or how the implicit role is determined, so what I'm describing here might not handle all cases nicely. I don't think this blows up the premise I'm describing, but I'll be thinking that feedback over and expect I'll be updating the content below in the future.

Intro

To explain the reasons that I mostly stopped using ByRole locators and started specifying elements, I want to zoom way out and talk about the various pieces of the puzzle. Here's a rough table of contents.

Background
What problems Testing Library is intended to solve
Why ByRole is too vague
What makes something accessible
What roles even are
The first rule of ARIA (Accessible Rich Internet Applications)
Semantic HTML, TypeScript, and using the platform
What we expect developers and testers to know when writing and testing their code
Situations where I still use ByRole
Why I love ByLabelText
The use of Test IDs
Overall conclusions

Since this argument is all in the details, this is a relatively long post, so I'm skipping some background on the underlying ideas of front-end testing and locating elements. If you're new to these topics, I have a 2022 post on CSS-Tricks called Writing Strong Front-end Test Element Locators that starts more from the ground up. But for now, I'm going to assume you have some familiarity with how front-end testing works and are interested in best practices for locating elements.

And of course, feel free to skip ahead to the conclusions if this is all familiar stuff.

Background

Testing Library provides a great set of accessibility-focused tools to locating elements when writing tests. It was created by Kent C Dodds, one of my longtime favorite educators on front-end and testing topics. Since the initial release in 2018 as "React Testing Library", it's grown in popularity beyond its React component testing roots. It now has plugins for all major test runners and JavaScript frameworks, and in 2022 some of its patterns were adopted in Playwright's test runner as the recommended way to locate elements.

I do almost all of my testing in Cypress (and for clarity: I work there). I've added Cypress Testing Library to multiple projects over the years, and it's my go-to for a lot of testing tasks, especially when testing forms, using the ByLabelText locator.

One thing I've noticed over time is that I've mostly stopped using the main workhorse of Testing Library, the ByRole locator. When I wrote some guidance for my team a few years ago, I recommended not using it in most cases. I also mention this as a quick caveat when I teach testing or accessibility topics (for example, it comes up in passing in a blog post about user-focused locators that I wrote last year).

I haven't given it much thought over the years, it's just one tool in the Testing Library toolbox and it has its place. My recommendation was driven by what I expect ByRole to accomplish. But recently I've noticed many strong recommendations to use ByRole as the best way to find an element in a test, citing the reason that this helps ensure the underlying application being tested is accessible.

The short version

My argument in this post will be that getByRole is not strict enough to ensure that a given accessible role is correctly implemented. Locating elements this way can create misleading scenarios where your tests imply that an element is accessible, even if it's not. We'll get into more detail on that below, but first I want to give you the TLDR on what I do instead.

The recommended Testing Library approach to find and test a button is this:

screen.getByRole('button', { name: /submit/i }).click()

What I prefer to do is locate an element by a combination of two things that, if they are changed, should fail a test:

The element's specific tag name - button or a, for example
The element's accessible name - "Save" or "Home", for example

In order to do this in Testing Library, for buttons and links, I would use the ByText locator, like this:

// get a "Submit" button
screen.getByText('Submit', { selector: 'button' })

// get a "Home" link in the navigation element
screen.getByText('Home', { selector: 'nav a' })

I do something similar using the following format of the contains command in Cypress. I tend to only look for another locator when this isn't going to get me the element I need:

// get a "Submit" button
cy.contains('button', 'Submit')

// get a "Home" link in the navigation element
cy.contains('nav a', 'Home')

The long version

The rest of this post is really just me trying to push at this from a few different angles and think everything through. I often find I have some opinion that I think is right, and then when I try to write about it, I realize I don't know the exact source of why I think something, or what things might just be assumptions. So it helps to document all the pieces, and it lets somebody else cross-check the ideas.

What is Testing Library for?

The basic premise of Testing Library is summed up by this quote from the home page:

The more your tests resemble the way your software is used,
the more confidence they can give you.

The problems Testing Library solves are described in two ways. The first is about avoiding testing implementation details to have better confidence that test failures matter:

You want tests for your UI that avoid including implementation details and rather focus on making your tests give you the confidence for which they are intended.

The second is about how not testing implementation details makes refactoring less painful:

You want your tests to be maintainable so refactors (changes to implementation but not functionality) don't break your tests and slow you and your team down.

The issue of test stability over time, and avoiding testing "implementation details", had previously often been addressed with a Test ID based approach, which maximizes the stability of tests. Testing Library's authors make the argument that using [data-testid="login"] style locators ignores important user-facing aspects of the elements being tested.

This is true. Test IDs intentionally and explicitly ignore the nature of the elements being used. We'll talk more about them at the end -- there's no disagreement there.

My issue with ByRole locators is the same as the problem with Test IDs, just in a narrower area. If either of these are your only locators and no test ever specifies the specific elements you expect, developers have too much wiggle room to break accessibility while keeping tests green.

`ByRole` locators are too forgiving for safe refactoring

The Testing Library query documentation says this about the ByRole locator:

This should be your top preference for just about everything. There's not much you can't get with this (if you can't, it's possible your UI is inaccessible).

The Playwright documentation puts it in similar terms:

We recommend prioritizing role locators to locate elements, as it is the closest way to how users and assistive technology perceive the page.

Both give examples of doing something like this:

screen.getByRole('button', { name: /submit/i }).click()

The underlying idea I take from this that if an element passes the "role check", it's likely to be accessible: locating an element by its role is intended to achieve the goal of safe refactoring - where "changes to implementation but not functionality" do not break your tests.

However, even though a role is part of an element's "identity" in an accessibility sense, it's not enough. We could refactor an accessible component in a way that fully breaks all accessibility, but still have a green test that uses getByRole. Many implementation changes that leave role intact are actually critical changes in functionality.

Here's an example of this kind of refactoring:

What makes a button accessible?

HTML elements all have implicit roles that describe the nature of the element to people with disabilities, accessing the page using assistive technology. Buttons have the button role, anchor tags with href attributes have the link role, input elements with type text have a textbox role and so on.

When it comes to interactive elements, these roles are not usually useful by themselves. What is useful about them is that they represent an overall contract that element will fulfill. To use the button element as an example, here is at least a partial list of the accessibility "contract" that browser vendors fulfill for a button:

It has the implicit role of button so that users know what it is - a screen reader, for example, would announce it as a button
It has a default browser button style to visually distinguish it from surrounding content
It is placed in the tab order of the document and can be focused with the keyboard
When focused it receives a focus outline
It can be activated with a mouse click, or the enter key, or the space bar
If placed inside a form element, activating the button by any of these methods will submit the form (because its default type is submit)
It will receive specific styling, customizable by the user, in Windows High Contrast Mode (used by people with certain visual disabilities)
It has default browser styles for hover and "pressed" states

That's a lot of behavior that the browser implements for a humble button. The browser also manages this button's presence in the Accessibility Tree, which is how assistive technology like a screen reader can announce the button's existence, what it does, and how the user understands what actions are possible.

And here's what may be surprising: the browser only does this stuff for real button elements. Adding role="button" to a div or a span does not trigger any of this in the browser. If role="button" is used on say, a div with an onClick handler, the entire contract of a button must be implemented by the developer, using extra HTML attributes, CSS rules, and JavaScript listeners. And even then, there will still be gaps - especially in Windows High Contrast Mode, which ignores role="button".

This is the main reason I avoid ByRole locators most of the time. Let's look at this locator again:

screen.getByRole('button', { name: /submit/i }).click()

This will absolutely find a true button element, because of its implicit role, but it will also have no problem finding a div with role="button".

Since these two elements can be very different from an accessibility point of view, it does not seem at all safe to depend on ByRole locators during a UI refactor. Accessibility can change radically depending on the underlying HTML, CSS, and JavaScript, even when roles are preserved during what is intended to be passive refactoring.

So, in the cases where there is a semantic HTML element available, I prefer to just specify that element, and that way the tests will fail if somebody refactors to a pattern that uses non-semantic elements to recreate existing HTML functionality.

This element-level approach also nudges developers towards accessibility success, by forcing them to either follow the first rule of ARIA in their refactoring, or modify the test code if they change the implementation. That means that in a pull request, for example, a reviewer would see unexpected test changes resulting from a refactor (where tests shouldn't change) and hopefully ask why semantic HTML was no longer being used. There's at least an extra layer of protection there.

The first rule of ARIA

ARIA stands for Accessible Rich Internet Applications and it's a specification that teaches us how to create online experiences that are likely to be functional for people with disabilities.

The first rule of ARIA is:

If you can use a native HTML element HTML51 or attribute with the semantics and behavior you require already built in, instead of re-purposing an element and adding an ARIA role, state or property to make it accessible, then do so.

The reason for this is to take advantage of what the browser provides, which is highly likely to work correctly with assistive technology, instead of doing a custom implementation of the same functionality.

One way to think about this is: the absolute best you can do for users when implementing a custom button, is to faithfully recreate everything the browser already does in the "button" contract, except your version won't support High Contrast Mode.

There is no value to writing and owning code that reimplements browser functionality - your company doesn't need it, you don't need it, and your users definitely don't need it.

Related practices

A lot of Testing Library advocates are also fans of other good practices, like using explicit types with TypeScript.

A ByRole locator essentially says that there is only one required property of an element, and everything else about its accessibility contract is optional - the test will accept whatever has that one single property. Specifying the correct semantic element, on the other hand, is a way to force the test to only except something where the browser is implementing the full contract. Following the first rule of ARIA in our tests, as well as our code, gives us the most "type safety" in this sense.

The alternative solution is to assert all of the expected accessibility behaviors in your tests, in addition to locating the element by its role, which is absolutely the way to go if you own the code that implements that functionality.

Another good practice is in web development is "using the platform", not fighting it. The first rule of ARIA perfectly aligns with this: using correct semantic elements allows browsers and operating systems to meet the needs of disabled users, and provides the best compatibility with assistive technology. When people say the web is "accessible by default", semantic HTML is at the heart of this statement.

Developers can still override things and break semantic elements of course, but semantic HTML is still a better foundation and more likely to be accessible. So having our tests depend on this aspect of code seems worth it.

What we expect developers to know

Because of the first rule of ARIA, I don't see any valid use case for role="button" in code. We should only see the role attribute being used for custom widgets that do not have native HTML equivalents.

The Testing Library docs point out something similar to the first rule of ARIA, but don't quite make the same conclusion:

Please note that setting a role and/or aria-* attribute that matches the implicit ARIA semantics is unnecessary and is not recommended as these properties are already set by the browser

This recommendation makes clear that it is expected and desired that developers know they should prefer maintaining the native semantics of HTML. Since they need this knowledge to choose the correct accessible implementation of a given role, we should take the mystery out of it and just specify the element that should be there in our tests. Then there is no need to hope that future developers know all the same nuances about accessibility. They can learn them when their non-semantic HTML causes a test failure.

When I still use `ByRole`

I'll take a ByRole selector any time for roles like menu, menuitem, tabpanel, treegrid... there are lots of useful accessible roles that are not yet implemented in standard HTML. Selecting them by role is a great idea, especially as it allow you to refactor into semantic HTML without breaking a test, when elements with those roles do make their way into browsers.

There are also some rare cases where ARIA roles are needed, even when there is overlap with semantic HTML elements. This might happen when working around some specific browser bug with the native element, for example, where owning the implementation of the accessibility contract is worth it because it lets you do something you can't do otherwise.

Why I love `ByLabelText`

OK, enough about why ByRole is not my thing. Let's talk about some good stuff. Here's my favorite way to locate a form field in Cypress:

cy.findByLabelText('Username').type('myusername')

This is not built in to Cypress -- I use the Testing Library plugin. It's elegant, readable, concise, and requires form fields to have a specific programmatically associated label present, which is also needed for assistive technology.

If I wasn't using Testing Library, I might define a custom command that knows how to find labels in my application, executing something like this:

cy.contains('Username').next('input').type('my-username')

But this depends on knowing the exact DOM structure, and here we don't care about that. There are a lot of correct ways to label a form input, it doesn't matter too much which way is used. One good thing is that here we are specifying a real input element, but Testing Library can do that too if we want to, using the selector option:

cy.findByLabelText('Username', {
    selector: 'input'
  }).type('myusername')

This option is explained in the docs:

If it is important that you query a specific element (e.g. an ) you can provide a selector in the options.

This is also presumably why ByText also takes a selector option. Even though we want to avoid testing implementation details when they don't matter, Testing Library has built-in options so that you can still specify them when they do matter.

In general the risk of developers creating custom inputs built out of divs and spans is pretty low, so I don't do that very often in practice. But it might be useful to use this pattern at the component test level, to give the developers instant feedback if they decide to get creative and reimplement an input out of a custom div.

So why don't we pass this selector option to our ByRole locators, where the risks of using non-semantic elements are higher? Primarily because ByRole doesn't support a selector option. That's for a pretty good reason as far as I can tell, since ByRole is specifically intended to avoid specifying the implementation.

It would be pretty pointless and circular to write code that looked like this:

getByRole('button', {
  name: /submit/i,
  selector: 'button' // ⚠️ not possible, just an example
}).click()

In the above format, the selector property would just override the role. On the other hand, in the ByLabelText scenario, since many elements can have similar labels, passing a selector option to further narrow what is acceptable makes more sense. You could have a Shipping and a Billing form on the same page with the same labels. You could have a select and an input with the same label text.

Or you might want to pass something like a [data-testid="header"] input selector if there are two instances of similar forms on the same page.

Oh yeah, what about `data-testid`?

As a big believer in accessibility, and a longtime user of Testing Library, I want to take a second to stick up for the good old Test ID pattern, which still has its place and can be used in conjunction with specifying the element, or the role, or anything else.

I strongly believe you need to cover the accessibility of your app with automated tests, and that means specifying the parts of the DOM that matter. The HTML we ship to users is our product and we should all pay attention to the actual code we ship. There is an enormous, expensive, complicated system of servers, databases, and various tubes often involved in sending it from us to our users. We might as well send the right HTML.

How I feel about Test IDs is this: if you've already specified the correct HTML output in one place, for example in component tests, then it's fine to use Test IDs elsewhere if you want.

Especially if you have a separate QA team, not as familiar with what the "right" HTML and roles are, we don't need to block them from workflows using generic helper locators, as long as there are tests owned and run by the developers writing code that will fail if HTML accessibility is broken. You must make sure your something in your system will fail if accessibility regressions happen.

On a personal level I'd be happy if everything went red when accessibility broke, because it's a critical thing for me. But as long as accessibility is specified, tested, and owned, then testing some workflows using Test IDs might actually have some upsides for certain teams.

It's also the case that many applications are not built in an accessible way where roles, or even elements themselves, are appropriate for their functionality. Those apps still need to be tested, and if it's not easy or possible to remediate things on the fly, a generic Test ID is a better choice than something that, say, specifies the current wrong role that something happens to have, and then causes confusion.

Test IDs also come in handy when disambiguating different instances of similar elements, and in a few other situations. It's just important to know what they do and do not cover.

I'm hoping to make a PR to the Cypress Best Practices documentation, which already endorses both Test IDs and Testing Library, to get into some of this kind of detail about how to choose the approach. But as you can see, what I've got here currently long and unwieldy. I'm hoping writing this post will help me form a much shorter proposed update to the docs.

If you've ready this far: thanks! And please let me know if you have thoughts or feedback. Am I missing something? Is the ByRole locator awesome in a way I'm not accounting for?

Conclusion

Here's the understanding I'm working with at the moment:

Semantic HTML, when available, is always preferred over custom role attributes for accessibility. This is the first rule of ARIA.
While users do perceive the role of an element, what they they depend on is its entire contract, so a ensuring a role exists is not enough to guarantee behavior doesn't change in refactors.
This means we should not use a ByRole locator if the role we are selecting on can be fulfilled by a native HTML element.
Instead we should specify the HTML element itself in our tests. It will help developers do the right thing, and provide better feedback when refactoring, giving you one extra layer of resistance to accessibility bugs.
We can do this in Testing Library by passing a selector option to our ByText and ByLabelText locators. And we can likely do something like it in our framework of choice even if Testing Library is not available, such as with cy.contains('selector', 'text').

Writing tests this way aligns your tests and your development practices with the first rule of ARIA, which will give you a great foundation for building accessible apps and avoid some common accessibility pitfalls that trap even the most well-intentioned of us.

Last note: Testing Library is still great and I'll still add it to every project to augment whatever I'm doing.

DEV Community: Mark Noonan

Skip tests that pass on the current build: Re-run optimization in Cypress Cloud

Why do teams re-execute failed Cypress runs?

How much time is saved with Re-run optimization in Cypress Cloud?

Choose spec-level or test-level reruns in Cypress Cloud

Controlling the re-run behavior in Cypress Cloud

Turn it on

Build custom workflows with Cypress Cloud Webhooks

Integration support in Cypress Cloud

Why webhooks?

When to use a webhook

Get started

What's next

Open Source Accessibility Plugins in Cypress

Cypress Voice Plugin

Links

Cypress Runner Themes

Links

Wick-A11y

What comes next

Why I rarely use `getByRole`: Testing Library and the first rule of ARIA

Intro

Background

The short version

The long version

What is Testing Library for?

ByRole locators are too forgiving for safe refactoring

What makes a button accessible?

The first rule of ARIA

Related practices

What we expect developers to know

When I still use ByRole

Why I love ByLabelText

Oh yeah, what about data-testid?

Conclusion

`ByRole` locators are too forgiving for safe refactoring

When I still use `ByRole`

Why I love `ByLabelText`

Oh yeah, what about `data-testid`?