DEV Community: Junior Diaz

Automating Web Accessibility Testing: Combining axe-core and Playwright

Junior Diaz — Fri, 19 Jun 2026 03:20:02 +0000

Introduction

In the previous article of this series, we covered how to detect unintended visual changes with visual regression testing in Playwright. This time we're tackling a different kind of bug that rarely shows up in a traditional functional test: accessibility violations.

Web accessibility isn't just a legal requirement — it's a fundamental principle of inclusive design. As an application grows, manually testing for accessibility compliance becomes impractical and error-prone. In this article we build an automated testing framework using Playwright and axe-core, and run it against AutoCatalog, my car catalog demo built specifically for this series (Next.js + TypeScript).

By the end of this article, you'll have a helper you can drop into your own project and start using right away.

Why Automate Accessibility Testing?

Adding automated accessibility testing to your workflow brings several concrete advantages:

Early detection: identify and fix issues before they reach production, significantly reducing remediation costs.
Consistent standards: automated tests apply the same criteria across every page and feature.
Regression prevention: a UI change that breaks accessibility gets caught before the merge, not after.
Compliance documentation: generated reports serve as evidence of WCAG 2.0/2.1 Level A and AA compliance efforts.
Developer empowerment: devs get immediate feedback on accessibility issues without needing to be accessibility experts.

The Tech Stack

Playwright

Playwright is a modern end-to-end testing framework with multi-browser automation capabilities. For accessibility testing it brings:

Reliable execution across Chromium, Firefox, and WebKit.
Built-in waiting mechanisms that ensure the page is fully loaded before analysis.
Natural integration with the rest of the suite (same fixtures, same page objects).
Real browser engine interaction, making tests reflect the DOM a user actually sees.

axe-core

Developed by Deque Systems, axe-core is the industry-standard engine for accessibility testing. It detects WCAG 2.0, WCAG 2.1, and other accessibility standard violations with very few false positives. Among what it identifies:

Missing alternative text on images
Insufficient color contrast
Incorrect heading hierarchies
Keyboard navigation issues
Missing form labels
Incorrect or missing ARIA attributes

@axe-core/playwright

This package connects Playwright and axe-core with a fluent API designed specifically for Playwright's Page object. It handles injecting axe-core into the browser context and extracting results, so you can focus on the test logic rather than the integration mechanics.

Architecture Overview

The accessibility testing framework relies on three layers:

Utility layer: reusable functions for analyzing pages and generating reports.
Test layer: concrete test cases for each page, modal, or state.
Report layer: transforms raw axe-core results into readable HTML reports, with detail on each violation.

Implementation: Building the Utility Layer

Core Analysis Function

analyzeAccessibility is the main interface for running the analysis:

import { Page } from '@playwright/test';
import AxeBuilder from '@axe-core/playwright';
import type { AxeResults } from 'axe-core';

export interface A11yAnalyzeOptions {
    tags?: string[];
    rules?: string[];
    disableRules?: string[];
    include?: string | string[];
    exclude?: string | string[];
}

export async function analyzeAccessibility(
    page: Page,
    options: A11yAnalyzeOptions = {}
): Promise<AxeResults> {
    const {
        tags = ['wcag2a', 'wcag2aa'],
        rules = [],
        disableRules = [],
        include = [],
        exclude = []
    } = options;

    const builder = new AxeBuilder({ page }).withTags(tags);

    if (rules.length > 0) {
        builder.withRules(rules);
    }

    if (disableRules.length > 0) {
        builder.disableRules(disableRules);
    }

    // Include specific elements/regions for testing
    if (include.length > 0 || (typeof include === 'string' && include)) {
        const includeSelectors = Array.isArray(include) ? include : [include];
        includeSelectors.forEach(selector => {
            builder.include(selector);
        });
    }

    // Exclude specific elements/regions from testing
    if (exclude.length > 0 || (typeof exclude === 'string' && exclude)) {
        const excludeSelectors = Array.isArray(exclude) ? exclude : [exclude];
        excludeSelectors.forEach(selector => {
            builder.exclude(selector);
        });
    }

    return await builder.analyze();
}

Key design principles:

WCAG compliance by default: the default tags are ['wcag2a', 'wcag2aa'], so without configuring anything you're already covering the baseline.
Flexible scoping: include and exclude let you test specific components in isolation.
Builder pattern: delegates to AxeBuilder for more complex analysis scenarios without reinventing the wheel.

💡 On the tags default: ['wcag2a', 'wcag2aa'] isn't an example value — it's the project's compliance target, defined once here in the helper. No need to repeat it in every test; you'd only override it when a specific test needs to deviate from that baseline (for example, to validate a particular rule with rules).

Report Generation

generateA11yReport transforms raw axe-core results into an HTML report:

import { createHtmlReport } from 'axe-html-reporter';
import * as fs from 'node:fs';
import * as path from 'node:path';

export interface A11yReportOptions {
    projectKey?: string;
    outputDir?: string;
    reportFileName?: string;
}

export function generateA11yReport(
    results: AxeResults,
    options: A11yReportOptions = {}
): void {
    const {
        projectKey = 'playwright-a11y',
        outputDir = path.join('test-results', 'a11y-reports'),
        reportFileName = 'a11y-report.html'
    } = options;

    // Ensure the output directory exists
    fs.mkdirSync(outputDir, { recursive: true });

    createHtmlReport({
        results,
        options: {
            projectKey,
            outputDir,
            reportFileName,
        }
    });
}

Each report includes the violation description, affected elements, suggested fixes, and links to the relevant guideline — very handy for sharing the report directly with a dev without having to translate axe-core's raw output.

Convenience Wrapper

analyzeAndReport combines analysis and reporting in a single call:

export interface A11yOptions extends A11yAnalyzeOptions, A11yReportOptions { }

export async function analyzeAndReport(
    page: Page,
    options: A11yOptions = {}
): Promise<AxeResults> {
    const { tags, rules, disableRules, include, exclude, projectKey, outputDir, reportFileName } = options;

    const results = await analyzeAccessibility(page, { tags, rules, disableRules, include, exclude });
    generateA11yReport(results, { projectKey, outputDir, reportFileName });

    return results;
}

Test Implementation Patterns

Full-Page Tests

A full-page accessibility test ensures the entire view meets accessibility standards:

test('Product Management - Default page - Accessibility validation', async ({ page }) => {
    await test.step('Validate accessibility', async () => {
        const results = await analyzeAndReport(page, {
            projectKey: 'autocatalog-a11y',
            reportFileName: 'manage-default.html',
        })
        expect(results.violations).toEqual([])
    })
})

If the page has accessibility violations, this test will fail — and the HTML report generated at manage-default.html details each one with its selector, description, impact, and suggested fix.

Modal Dialog Tests (with `include`)

Modals are a good candidate for automated accessibility testing because they concentrate several criteria axe-core can verify statically: dialog ARIA attributes (role, aria-label, aria-modal), labels associated with each form field, and visible focus or contrast indicators. But there's a problem with testing a modal through a full-page scan: it picks up everything else that's broken on the page too.

test('Product Management - Add Product modal open - Accessibility validation', async ({ page }) => {
    await test.step('Open Add Product modal', async () => {
        await managePage.openAddProductModal()
        await managePage.expectAddProductModalVisible()
    })

    await test.step('Validate accessibility', async () => {
        const results = await analyzeAndReport(page, {
            include: "[role='dialog']",
            projectKey: 'autocatalog-a11y',
            reportFileName: 'manage-add-product-modal.html',
        })
        expect(results.violations).toEqual([])
    })
})

📸 3 violations detected — full-page scan

📸 0 violations — scoped to [role='dialog']

Best Practices and Patterns

Test Organization

Organizing accessibility tests to reflect the application's structure helps a lot when reading reports and running subsets:

Use describe blocks to group by page/feature.
Name each test so it's clear which state or component is being validated.
Keep separate test files per feature.
Use tags like @A11y and feature-specific tags (@ProductManagement) for selective execution.

Example structure, based on AutoCatalog's real suite:

test.describe('@ProductManagement @A11y - Product Management accessibility tests', () => {
    test('Product Management - Default page - Accessibility validation', /* ... */)
    test('Product Management - Add Product modal open - Accessibility validation', /* ... */)
})

The same pattern repeats for HomePage (with tag @Home @A11y) and CartPage (@Cart @A11y), each with their own relevant states.

Selective Test Execution

The tag system lets you run specific subsets depending on what you need to validate:

# Run all accessibility tests
npx playwright test --grep "@A11y"

# Run only Product Management tests
npx playwright test --grep "@ProductManagement"

# Run only Home tests
npx playwright test --grep "@Home"

This gives you the flexibility to run everything on each PR, or split by feature for parallelization.

Understanding the Limits of Automated Testing

Automated accessibility tests catch a lot of issues, but not all of them. Automated tools typically identify between 30% and 50% of accessibility problems.

Issues That Require Manual Testing

Content quality: whether the alt accurately describes the image and its context.
Focus order: whether the tab order makes logical sense.
Cognitive accessibility: whether the content and navigation are comprehensible.
Meaningful sequence: whether content retains meaning when read linearly.
Complex interactions: whether multi-step flows work well with assistive technology.
Real screen reader experience: how content is announced and navigated.
Keyboard shortcuts: whether custom shortcuts conflict with those of assistive technology.

A Comprehensive Strategy

Automated testing (30-50% coverage): catches common violations early.
Manual testing with keyboard and screen reader (adds 20-30%).
Research with real assistive technology users (remaining issues + UX insights).
Ongoing developer education.

💡 Need to complement with visual testing? Some WCAG criteria, like 2.4.7 (Focus Visible), require visual verification that axe-core can't fully automate. Article 3 of this series covers how to validate focus indicators with visual regression in Playwright.

Tracking Progress

Some metrics worth keeping an eye on as the suite grows:

Violation trends: count and severity over time.
Test coverage: what percentage of the app is covered by accessibility tests.
Remediation time: how long it takes for a detected violation to get fixed.
Prevention rate: violations caught in PR vs. those that reach production.
Manual findings: issues automation missed (they tell you where the real gaps are).

Suite Structure

Here's how the accessibility suite is organized in autocatalog-testing:

tests/
├── accessibility/
│   ├── ProductManagementA11yTest.ts
│   │   └── @ProductManagement @A11y
│   │       ├── Default page
│   │       └── Add Product modal (include: [role='dialog'])
│   ├── HomeA11yTest.ts
│   │   └── @Home @A11y
│   └── CartA11yTest.ts
│       └── @Cart @A11y
│
├── utils/
│   └── AccessibilityHelper.ts
│       ├── analyzeAccessibility()
│       ├── generateA11yReport()
│       ├── analyzeAndReport()
│       └── generateReportFileName()
│
└── pageObject/
    ├── POManager.ts
    ├── HomePage.ts
    ├── ManagePage.ts
    └── CartPage.ts

Common Troubleshooting

Tests timing out

await page.waitForLoadState('networkidle');
await page.waitForSelector('.main-content');
const results = await analyzeAndReport(page, options);

Flaky tests due to dynamic content

await page.waitForSelector('[data-testid="product-table"]', { state: 'visible' });
const results = await analyzeAndReport(page, options);

Third-party components with many violations

If you're working with a third-party component you can't fix and need to exclude it from the analysis, @axe-core/playwright supports exclude for that (see the official docs). AutoCatalog doesn't have that scenario, so we don't cover it in detail here — but it's good to know the option exists if you need it.

Fun Facts

Three things I learned while building this suite that change how you read (or configure) an axe-core analysis:

axe-core reads the accessibility tree, not the DOM. That's why two tools can scan the same page and report different things without either being "wrong": axe-core evaluates the model that screen readers and other assistive technologies actually consume, while tools like WAVE read the DOM more directly. An element with aria-hidden="true" may have DOM-level issues (missing label, contrast problem) that WAVE flags — but axe-core correctly ignores it because for any assistive technology, that element simply doesn't exist.
color-contrast doesn't evaluate disabled elements. axe-core completely excludes from this rule any element with disabled or aria-disabled="true" — it doesn't mark it as "pass", "fail", or "incomplete", it simply doesn't evaluate it, following the exemption WCAG 1.4.3 makes for inactive components. A disabled button with terrible contrast will never show up in the report.
axe-core's tags go beyond WCAG. Playwright's docs only show four (wcag2a, wcag2aa, wcag21a, wcag21aa), but axe-core supports many more — wcag22aa, best-practice, section508, EN-301-549, among others. It's not something you'll need to touch every day, but the available surface is wider than it first appears, in case you ever need to align your tests with a standard other than WCAG.

The tool isn't being lenient — it's being precise. But precision only helps if you know what you're measuring.

Conclusion

Automated accessibility testing with Playwright and axe-core is a solid foundation for building more inclusive applications, and it catches a good share of problems well before they reach production.

Key takeaways:

Start small, iterate: begin with the most-used flows and expand coverage over time.
Combine automated and manual: automation covers 30-50%; manual testing is still necessary.
Use include to isolate the component you actually want to test, without the result getting mixed up with the rest of the page.
Understand what the tool measures — accessibility tree vs. DOM, color-contrast exclusions on disabled elements — before interpreting (or trusting) its results.
Use tags for selective execution, so you can run the full accessibility suite or just the subset you care about.

Does your team have automated accessibility coverage? If you want to explore how to implement this kind of testing in your project, tell me about your team here.

Additional Resources

This is the second article in the series — coming up next: **WCAG 2.4.7 Focus Visible: Visual Regression Testing with Playwright**

Visual Regression Testing with Playwright: Detecting UI Changes Automatically

Junior Diaz — Thu, 11 Jun 2026 02:38:20 +0000

Introduction

When testing web applications, we tend to focus on functionality — but how do we ensure the user interface looks correct after every deployment?

That's where visual testing comes in.

Visual testing automatically compares UI screenshots between test runs, helping us detect unexpected layout changes, missing elements, or style regressions. A padding change, a button that shrank, a component that shifted two pixels — the kind of thing that slips through code review but that your UX and QA team needs to catch before it reaches production.

Although dedicated tools like Percy by BrowserStack offer advanced visual comparison capabilities, they come with a screenshot limit. Playwright's built-in functionality has no such direct cost — snapshots live in your repository. The "cost" is disk space and Git history size, both of which you can manage with a proper storage strategy. For teams and clients that prioritize open source tools, this is a compelling argument: you get visual comparison, historical evidence, multi-browser and multi-breakpoint support, all within the same tool you already use for functional testing.

In this article I'll show you how to set up visual regression testing with Playwright using real examples from AutoCatalog, a car catalog I built in Next.js + TypeScript to experiment with these patterns.

By the end, you'll be able to:

Capture baseline screenshots of key UI components
Automatically compare them in future test runs
Detect unwanted visual changes early in your CI pipeline

⚙️ Prerequisites

Before we start, make sure you have:

Node.js (LTS version recommended)
A Playwright project already set up:

pnpm create playwright

Basic knowledge of Playwright tests and the Page Object Model (POM) pattern

Setting Up Visual Tests

1. Basic Screenshot Assertions

Playwright includes built-in support for visual comparisons via expect(page).toHaveScreenshot() or expect(element).toHaveScreenshot().

The first time you run a test, Playwright saves a baseline image. In future runs, it compares the new screenshot against the baseline and flags any differences.

To illustrate the real value of this: imagine someone modifies the size or padding of the "Add Product" button — a small change that easily goes unnoticed in code review. Visual regression testing catches it immediately.

test('Product Management - Default layout page', async ({ page }) => {
    await test.step('Capture full-page baseline', async () => {
        await expect(page).toHaveScreenshot('manage_default_layout.png')
    })
})

If the layout changes, Playwright generates a diff image and the report includes an interactive slider to compare before and after:

📸 The Playwright report slider lets you visually compare the baseline against the current state — ideal for catching subtle spacing or size changes.

2. Component-Level Visual Testing

Instead of capturing full pages, you can focus on specific components or modals. This makes tests more stable: a header change won't break the modal test.

test('Product Management - Add Product modal default layout matches baseline', async ({ page }) => {
    await test.step('Open Add Product modal', async () => {
        await managePage.openAddProductModal()
        await managePage.expectAddProductModalVisible()
    })

    await test.step('Capture modal component screenshot', async () => {
        const productModal = managePage.modalTitle.locator('..')
        await expect(productModal).toHaveScreenshot(
            'add_product_modal_default.png'
        )
    })
})

The screenshot scope is the modal itself — not the full page. This way, the test verifies that the modal's layout, spacing, and elements stay consistent without depending on the rest of the UI.

📸 Snapshot of the Add Product modal — the test verifies that the layout, spacing, and modal elements remain consistent.

3. Capturing Snapshots at Key UI States

Visual tests are most effective when you capture relevant interface states, not every interaction.

The home carousel: the dynamic content problem

We verify that the home page layout looks correct without noise generated by animations. The carousel displays images in random positions on each run — without mask, the test fails constantly even when the layout is perfectly fine.

test('Home - Full page screenshot with carousel masked — stable test', async ({ page }) => {
    await test.step('Wait for carousels to be visible', async () => {
        await homePage.expectCarouselsVisible()
    })

    await test.step('Capture full page with all carousels masked', async () => {
        const allCarousels = await homePage.getAllCarousels()
        await expect(page).toHaveScreenshot('home_with_mask.png', {
            mask: allCarousels,
        })
    })
})

👉 The mask: Locator[] option ignores areas that change dynamically, like the carousel.

Let's look at both scenarios:

📸 Without mask: the test fails because the carousel shows different images between runs — Playwright detects the differences and marks them in red.

📸 With mask applied: the dynamic areas are ignored and the test passes consistently.

4. Managing Baseline Images

The first time you run the tests, Playwright saves the baseline screenshots under:

/tests/screenshots/

If a visual change is intentional, update the snapshots with:

pnpm exec playwright test --update-snapshots

⚠️ Note for CI: Avoid running --update-snapshots automatically in your pipeline. Reserve this command for manual, reviewed updates. Consider creating a separate workflow or gating it behind an explicit confirmation.

To review differences:

Open the test-results folder
Compare the baseline, actual, and diff images
Approve or reject changes during code review

Best Practices

1. Organize Tests by Feature

Group visual tests logically using test.describe():

test.describe('@ProductManagement @Visual', () => {
    test.describe('@LayoutBaseline', () => {
        // Full layout test
    })

    test.describe('@ModalBaseline', () => {
        // Modal component tests
    })
})

2. Use Descriptive Screenshot Names

Name screenshots clearly to indicate what they're testing:

✅ manage_default_layout.png
✅ add_product_modal_masked_dropdowns.png
❌ screenshot1.png

3. Prepare Test State Properly

Make sure the application is in the correct state before capturing a screenshot:

test.beforeEach(async ({ page }) => {
    poManager = new POManager(page)
    managePage = poManager.getManagePage()
    await managePage.navigateToManage()
    await managePage.expectManagePageLoaded()
})

4. Use `test.step` with Snapshots

You can take snapshots inside test.step, which lets you capture multiple states within a single test. The upside: the test stays cohesive and the Playwright report shows exactly which step failed. The tradeoff: if one step fails, the following ones don't run — in some cases it's better to split them into separate tests.

test('Add Product modal — multiple states', async ({ page }) => {
    await test.step('Open modal', async () => {
        await managePage.openAddProductModal()
        await managePage.expectAddProductModalVisible()
    })

    await test.step('Capture default state', async () => {
        await expect(productModal).toHaveScreenshot('modal_default.png')
    })

    await test.step('Capture focus on submit button', async () => {
        await managePage.activateTabKeyboard()
        await managePage.focusModalSubmitButton()
        await expect(productModal).toHaveScreenshot('modal_submit_focus.png')
    })
})

5. Test Responsive Layouts

Verify that responsive layouts work correctly across different screen sizes:

test('Home - mobile layout', async ({ page }) => {
    await page.setViewportSize({ width: 375, height: 667 })
    await expect(page).toHaveScreenshot('home-mobile.png')
})

🧠 Troubleshooting Common Issues

Problem	Solution
Minor pixel differences between runs	Identify the root cause: use `mask` for dynamic content, `addStyleTag` for scrollbars, or explicit waits for elements still rendering. Avoid `maxDiffPixelRatio` — it accepts incorrect pixels as valid and can hide real regressions.
Dynamic content (carousels, timestamps, animations)	Use the `mask` option to exclude it from the snapshot
False positives in CI	Ensure consistent screen resolution and browser version across environments
Scrollbars generating diffs	Hide them with `addStyleTag` before the screenshot

⚠️ On maxDiffPixelRatio: While Playwright offers this option to tolerate minor pixel differences, use it with care. A value of 0.01 may seem small, but on a 1280×720 screen that's over 9,000 pixels — enough to hide a misaligned button or a color change. In most cases, the right fix is to eliminate the source of instability, not tolerate it.

🎯 Conclusion

By combining Playwright's functional tests with visual assertions, you can confidently detect and prevent unwanted UI changes.

This approach gives your QA and development team an extra safety net: ensuring that both how your app behaves and how it looks remain consistent over time.

Key takeaways:

Use toHaveScreenshot() to capture and compare UI states
Mask dynamic elements to get reliable diffs
Version your baselines and review changes through the code review process
Avoid updating snapshots automatically in CI without an explicit review
Use test.step to organize tests with multiple captures and get more granular reports

This is the first in a series on QA automation — coming soon:

Automating Web Accessibility Testing: Combining axe-core and Playwright
WCAG 2.4.7 Focus Visible: Visual Regression Testing with Playwright

Does your team have visual regression coverage? If you want to explore how to implement this kind of testing in your project, tell me about your team here.

DEV Community: Junior Diaz

Automating Web Accessibility Testing: Combining axe-core and Playwright

Introduction

Why Automate Accessibility Testing?

The Tech Stack

Playwright

axe-core

@axe-core/playwright

Architecture Overview

Implementation: Building the Utility Layer

Core Analysis Function

Report Generation

Convenience Wrapper

Test Implementation Patterns

Full-Page Tests

Modal Dialog Tests (with include)

Best Practices and Patterns

Test Organization

Selective Test Execution

Understanding the Limits of Automated Testing

Issues That Require Manual Testing

A Comprehensive Strategy

Tracking Progress

Suite Structure

Common Troubleshooting

Tests timing out

Flaky tests due to dynamic content

Third-party components with many violations

Fun Facts

Conclusion

Additional Resources

Visual Regression Testing with Playwright: Detecting UI Changes Automatically

Introduction

⚙️ Prerequisites

Setting Up Visual Tests

1. Basic Screenshot Assertions

2. Component-Level Visual Testing

3. Capturing Snapshots at Key UI States

The home carousel: the dynamic content problem

4. Managing Baseline Images

Best Practices

1. Organize Tests by Feature

2. Use Descriptive Screenshot Names

3. Prepare Test State Properly

4. Use test.step with Snapshots

5. Test Responsive Layouts

🧠 Troubleshooting Common Issues

🎯 Conclusion

📚 Further Reading

Modal Dialog Tests (with `include`)

4. Use `test.step` with Snapshots