TestFly Part 12: Playwright-Style Assertions in Selenium — not(), ARIA Snapshots & Strict Mode

TL;DR: TestFly 1.1.0 closes the gap with Playwright for Java's assertion API without leaving Selenium: LocatorAssert.not() negates the next check, hasRole() / hasAccessibleName() / hasAccessibleDescription() read the real accessibility tree instead of guessing from DOM attributes, matchesAriaSnapshot() compares a subtree against a Playwright-style YAML template, and opt-in locators.strict turns an ambiguous locator into a loud failure instead of a silent click on the wrong element.

---

Why Do Selenium Assertions Need a not() Method?

Negative assertions in classic Selenium/TestNG code usually mean one of two bad options: wrap the positive assertion in a try/catch and invert the result, or write a separate polling loop that waits for a condition to stay false. Both bury the real intent of the test in boilerplate.

TestFly's LocatorAssert now carries the same not() Playwright for Java has — it negates only the assertion directly after it in the chain:

import io.testfly.test.BaseTest;
import org.testng.annotations.Test;

import static io.testfly.assertion.Assertions.assertThat;

public class CheckoutErrorTest extends BaseTest {

    @Test
    public void testErrorBannerClearsAfterRetry() {
        open("/checkout");
        find("#pay-btn").click();

        assertThat(find("#error-banner")).hasText("Payment declined");

        find("#retry-btn").click();

        // Only the assertion right after not() is negated
        assertThat(find("#error-banner")).not().isVisible();
        assertThat(find("#spinner")).not().isAttached();
    }
}

As in Playwright, most negated assertions also pass when no matching element exists — not().isVisible() is satisfied whether the banner is hidden or gone from the DOM entirely. If the test needs the element to exist and just be in a different state, assert presence first: assertThat(status).isVisible().not().hasText("Error"). not() cannot be combined with the verified actions click(), fill() or clear().

What Are Deterministic Accessibility Assertions?

TestFly already shipped hasAriaLabel-style attribute checks. 1.1.0 adds assertions that read the computed accessibility tree the same way a screen reader would, through one bundled probe script — no axe-core injection, no LLM call:

@Test
public void testSubmitButtonIsAccessible() {
    open("/signup");

    assertThat(find("#submit"))
        .hasRole(Role.BUTTON)
        .hasAccessibleName("Create account");

    assertThat(find("#email"))
        .hasAccessibleDescription(Pattern.compile("(?i)required"));
}

hasRole(Role | String) checks the explicit role attribute or the implicit role of the HTML tag. hasAccessibleName and hasAccessibleDescription accept either an exact String (whitespace collapsed, same matching Locator.withName(...) uses) or a Pattern for a partial/case-insensitive match. Only the 36 standard ARIA Role values and visible text are reported; decorative images and hidden content are skipped, and Shadow DOM is not entered.

How Do ARIA Snapshots Work?

matchesAriaSnapshot() is TestFly's answer to Playwright's toMatchAriaSnapshot(): it captures the accessibility subtree under a locator and compares it against a YAML template — a partial, ordered constraint, not a full serialization diff.

@Test
public void testNavigationLandmarkStructure() {
    open("/dashboard");

    assertThat(By.tagName("nav")).matchesAriaSnapshot("""
        - link "Home"
        - link "Docs"
        - button "Account" [expanded=false]
        """);
}

Matching rules, in the style of Playwright:

Rule	Behavior
Role	must match exactly
Quoted name	case-insensitive substring match
/regex/ name	matched with find()
[attribute] / [level=2]	every template attribute must be present on the page; extra page attributes are ignored

A malformed template fails immediately with IllegalArgumentException, before waiting on the timeout. On failure, the assertion error includes both the template and the captured snapshot, so the diff is visible without re-running anything. To write a template in the first place, print what TestFly currently sees:

System.out.println(find("nav").ariaSnapshot());
// or, for the whole page:
System.out.println(ariaSnapshot());

and paste the printed YAML into the test, trimmed to only the nodes that matter.

How Does Strict Mode Prevent Silent Multi-Element Bugs?

By default, when a locator resolves to several visible elements, TestFly — like classic Selenium — silently clicks or reads the first one. That is exactly how a renamed CSS class or an A/B-tested duplicate banner turns into a test that passes against the wrong element for weeks before anyone notices.

Opt in per project with:

# testfly.yml
locators:
  strict: true

With locators.strict: true, an action on a locator matching more than one visible element — and not narrowed with first(), nth() or last() — now throws immediately:

Strict mode violation: css=".card .btn-buy" resolved to 3 visible elements;
use first()/nth()/last() or narrow the locator. Candidates: <button.btn-buy#card-1>, <button.btn-buy#card-2>, <button.btn-buy#card-3>

isVisible() and isEnabled() surface the same violation instead of silently checking the first match; count(), elements() and assertThat(...) (for counting/enumerating) are unaffected, since ambiguity is the expected case there. Narrow the locator explicitly once strict mode flags it:

find(".card .btn-buy").nth(1).click();          // second card, explicit
find(".card").filter(withText("Pro Plan"))
             .find(".btn-buy").click();          // narrowed by content instead

Hidden duplicates — a closed mobile nav sitting next to the desktop one, a hidden modal clone — are filtered out before the check runs, so responsive layouts don't trigger false strict-mode violations.

FAQ

Does not() replace the existing Selenium assertion methods?

No. not() is a modifier on LocatorAssert — it negates whichever assertion comes right after it in the same chain. All existing positive assertions are unchanged.

Do the accessibility assertions require axe-core or an external library?

No. hasRole, hasAccessibleName, hasAccessibleDescription and matchesAriaSnapshot use one bundled probe script that reads the browser's computed accessibility tree directly — no axe-core injection and no LLM call.

Will enabling locators.strict break existing tests?

It can, by design — that is the point. Any locator that was silently matching more than one visible element before now fails explicitly. The fix is the same one Playwright recommends: narrow the locator with first(), nth(), last(), or a more specific selector.

Does strict mode affect count() or elements()?

No. Those methods exist specifically to work with multiple matches, so strict mode does not apply to them — only to actions (click, fill, …) and the boolean checks isVisible()/isEnabled().

---

Originally published on hakangul.lovable.app.