Stop asking AI if your site is accessible. Ask it what to test by hand.

Give a current model a browser, a keyboard and axe, and it finds more accessibility bugs than axe does. It also flags plenty the human auditors never wrote down. In a September 2026 preprint that scored five models against human audit reports, the best agent setups caught 77% to 86% of known WCAG violations, where axe-core caught 36%. But 44% to 45% of what the agents flagged wasn't in the human reports, compared with 10% for axe. Some may be real bugs the auditors missed, but nearly half the flags need a person before anyone can act on them, while axe, wrong about one in ten, can fail a build on its own. So axe stays the gate, and a model sorts its output and tells me what to test by hand.

How much a scanner finds

The WebAIM Million 2026 found detectable WCAG 2 failures on 95.9% of the top million home pages, 56.1 errors per page on average. Low contrast text was on 83.9% of them and missing alt text on 53.1%. You don't need a language model to tell you an image has no alt text.

Deque, which makes axe, reported in 2021 that its automated tests covered 57% of issues by volume, from its own audits. In a 2017 GDS test on a page with 143 deliberate failures, 10 tools combined caught 71%, the best single tool about 40%.

The W3C's own guidance says "Web accessibility evaluation tools can not determine accessibility, they can only assist in doing so." A Lighthouse 100 only means its weighted automated audits passed.

Asking a model for an audit

The preprint, by Mishra and colleagues, isn't peer-reviewed. They took 24 pages from 11 scholarly platforms with existing Library Accessibility Alliance audit reports and checked each model's findings against them. The strongest setups were browser agents (GPT-5.5, Claude Sonnet 5, Gemini 3.1 Pro) with the DOM, screenshots, a keyboard and axe. Claude Opus 5 ran inside Claude Code, and Gemini 3.5 Flash and others got a single batch pass.

In the paper's terms, axe-core had precision 0.90 and recall 0.36. The reference browser-agent run (GPT-5.5) reached recall 0.86 at precision 0.56, a repeat run 0.88, and Claude Code 0.77 at 0.55 with the authors' parity prompt. The authors had seen 19 of the 24 pages while building the setup, so this isn't a clean held-out test, and a "correct" pass was inferred from what the human reports didn't mention.

A browser agent driving Playwright MCP can press Tab and tell you where focus landed, and in this preprint the browser agents caught 6 of 7 focus order failures from the human audits. The paper gives no false alarm rate per criterion and didn't test focus visibility at all. On navigation failures (focus, state, feedback), only 57.2% of the failures a DeepSeek V4 Flash browser agent reported in the A11yLTLNav preprint were confirmed, compared with 88.7% for the authors' rule-based checker, which uses no LLM, both on 31 generated clones of real sites. Nobody I found has measured a model judging what a screen reader announces. So focus order, focus visibility and screen reader output stay on the manual list. An agent may add to that list, never close it.

A six-step flow

Steps 1 and 6 run in CI. A model drafts the output of steps 2 and 5, and you check it. Steps 3 and 4 are done by hand. After a fix, go back to step 1.

1. Automated scan

Playwright with @axe-core/playwright, following the Playwright accessibility testing docs:

npm i -D @playwright/test @axe-core/playwright @types/node
npx playwright install chromium   # a fresh project has no browsers yet

page.goto('/') needs a baseURL, and a second project covers the mobile layout:

// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  use: { baseURL: 'http://localhost:3000' },
  webServer: {
    command: 'npm run start',
    url: 'http://localhost:3000',
    reuseExistingServer: true,
  },
  projects: [
    { name: 'desktop', use: { ...devices['Desktop Chrome'] } },
    { name: 'mobile', use: { ...devices['Pixel 7'] } },
  ],
});

One shared helper holds the scan configuration:

// tests/axe.ts
import type { Page } from '@playwright/test';
import AxeBuilder from '@axe-core/playwright';

export const scan = (page: Page) =>
  new AxeBuilder({ page })
    .withTags(['wcag2a', 'wcag2aa', 'wcag21a', 'wcag21aa', 'wcag22aa'])
    // The chat widget is the vendor's code, so it's excluded to keep this build
    // independent of their releases. It still ships on our page: it stays on the
    // manual checklist and in a ticket with the vendor.
    .exclude('#third-party-chat-widget');

// tests/a11y.spec.ts
import { test, expect } from '@playwright/test';
import fs from 'node:fs';
import { scan } from './axe';

test('home: initial state', async ({ page }, testInfo) => {
  await page.goto('/');
  await expect(page.getByRole('main')).toBeVisible(); // let the UI settle first
  const results = await scan(page).analyze();

  const file = testInfo.outputPath('axe-results.json'); // input for step 2
  fs.writeFileSync(file, JSON.stringify(results, null, 2));
  await testInfo.attach('axe-results', { path: file, contentType: 'application/json' });

  expect(results.violations).toEqual([]);
});

test('home: menu open', async ({ page }) => {
  await page.goto('/');
  await page.getByRole('button', { name: 'Open menu' }).click();
  await expect(page.locator('#menu')).toBeVisible();
  const results = await scan(page).include('#menu').analyze();
  expect(results.violations).toEqual([]);
});

Keep wcag21a. In axe-core 4.14 it holds one rule, label-content-name-mismatch (2.5.3 Label in Name), and it's easy to drop. The axe-core API docs list every tag, including EN-301-549.

analyze() scans the DOM as it is at that moment, so add a test per interactive state: an open menu, a modal, a form after a failed submit. Cover one URL per template, with the same tests in a for loop over ['/', '/products/example', '/cart', '/contact']. axe also returns an incomplete array for checks where it could not be certain. Those go on the manual list.

Starting from hundreds of violations?

On an existing site, toEqual([]) is a red build on day one, and red builds nobody can fix tend to get skipped. The Playwright docs suggest snapshotting violation fingerprints instead, so the test fails when the set changes. It replaces the two toEqual([]) tests above until the backlog is gone.

// tests/a11y-baseline.spec.ts
import { test, expect } from '@playwright/test';
import type { AxeResults } from 'axe-core';
import { scan } from './axe';

const fingerprints = (results: AxeResults) =>
  JSON.stringify(
    results.violations.map((v) => ({ rule: v.id, targets: v.nodes.map((n) => n.target) })),
    null,
    2,
  );

test('home: no new violations', async ({ page }) => {
  await page.goto('/');
  await expect(page.getByRole('main')).toBeVisible();
  const results = await scan(page).analyze();
  expect(fingerprints(results)).toMatchSnapshot('home-a11y.txt');
});

The first run fails and writes the snapshot. Commit it, and re-record with --update-snapshots after each fix. Snapshots are named per project and OS (home-a11y-desktop-linux.txt), so record them on the OS your CI runs.

Once the snapshot is down to minor and moderate issues, add a hard gate: filter results.violations to impact critical and serious and expect an empty list. If one rule floods the report, disableRules(['rule-id']) turns it off, with a ticket number in a comment next to it.

2. Triage the JSON with a model

One bug in a shared <Button> can show up as dozens of violations. The test writes its JSON to test-results/<test>-<project>/axe-results.json, and Playwright clears that folder on every run, so copy the file out. Most of it is passing checks, so trim it:

jq '{violations: [.violations[] | {id, impact, help,
      nodes: [.nodes[] | {target, html: .html[0:200]}]}],
    incomplete: [.incomplete[] | {id, help, nodes: [.nodes[] | {target}]}]}' \
  axe-results.json > axe-slim.json

Give the model the trimmed JSON and the source of the affected components, and constrain it hard. Check first that you're allowed to send that code to the model.

Input: axe-slim.json below holds the `violations` and `incomplete` output of
axe-core (tags wcag2a, wcag2aa, wcag21a, wcag21aa, wcag22aa) for <URL>,
plus the source of the affected components.

Rules:
- Use only the axe output and the code I gave you. Do not add issues.
- Quote the exact `target` selectors from the input for every group.
- Mark anything you infer (a root cause, which component renders a node) GUESS.
- Do not say whether the page is accessible or compliant, and do not write fixes yet.

Tasks:
1. Group violations by likely root cause and component.
2. Order the groups by axe impact, then by node count. Put anything in
   checkout, sign-in or contact forms first.
3. For each group, explain the WCAG success criterion in two sentences.
4. Turn the `incomplete` items into a manual checklist for this page.
5. From the element types in this code (forms, dialogs, carousels, media),
   list what I need to test by keyboard and screen reader.

Output: a markdown table with columns
group | rule ids | target selectors | node count | impact | component | GUESS?
then the lists from tasks 4 and 5.

The quoted selectors let you confirm in seconds that a group is real.

3. Keyboard pass

Unplug the mouse, or at least stop touching it. Safari on a Mac needs Safari > Settings > Advanced > "Press Tab to highlight each item on a webpage" turned on first (Apple). From WebAIM's keyboard guide:

• Focus is visible on every interactive element. outline: none with no replacement fails.
• Tab and Shift+Tab follow the visual layout, with no traps.
• Modals keep focus inside, close on Esc and return focus to the button that opened them.
• Enter works on links, Enter and Space on buttons (including anything with role="button"), Space on checkboxes, arrow keys inside radio groups and tabs.

Add these:

• The first Tab stop skips to the main content.
• No sticky header, cookie banner or chat bubble covers the focused element completely (2.4.11, new at AA in WCAG 2.2).
• After a client-side route change, focus lands somewhere sensible.
• Tooltips and mega menus can be dismissed without moving focus and stay put while the pointer moves onto them (1.4.13).
• Text over photos and gradients passes contrast. axe can't measure that, so use a colour picker.

4. Screen reader pass

NVDA is free and runs on Windows 10 and later. From WebAIM's NVDA shortcuts: H, D and F jump between headings, landmarks and form fields, NVDA + F7 opens the Elements List and NVDA + Space switches between browse and focus mode. If pressing H types an "h" into a field, you're in focus mode.

On a Mac, VoiceOver toggles with Command-F5 (Apple's guide), VO + U opens the Rotor, and per WebAIM it works best in Safari.

Listen for:

• Headings and landmarks that form a sensible outline
• Link text that makes sense out of context
• Each field's label, required state and error, plus focus on the first error or an error summary after a failed submit
• States: expanded or collapsed on disclosure buttons, selected on the active tab
• A name on every icon-only button (cart, close, search)
• Dynamic messages like "added to cart" announced at all

5. Fixes drafted by a model, reviewed by you

Give the model one group from step 2 with its component code, and ask for a fix plus a Playwright test that fails if the bug comes back. In Microsoft's A11y LLM Eval, models writing UI code passed its automated checks 12% of the time with no accessibility instructions and 86% with a skill that makes the model review its own output. Those are automated checks, so read the diff yourself, the way you'd read a junior's pull request, ARIA first. The ARIA Authoring Practices have a section titled "No ARIA is better than Bad ARIA". Check:

• Would a native <button>, <a href>, <label> or <dialog> do the job?
• Does every aria-label contain the visible text? 2.5.3 Label in Name exists because "people who operate with voice interaction use the visible labels in their commands". A button that shows "Buy" with aria-label="Add product to cart" fails it.
• Is aria-hidden="true" on something focusable? Then it's skipped by screen readers but still in the focus order.

After the fix, rerun axe and repeat the keyboard and screen reader checks for that component.

6. Regression test back in CI

Every bug you found by hand gets its own test, because axe won't notice it coming back. For the menu, in tests/menu.spec.ts:

import { test, expect } from '@playwright/test';

test('menu closes on Escape and returns focus', async ({ page }) => {
  await page.goto('/');
  // If the label changes to "Close menu" when open, locate it by test id instead
  const trigger = page.getByRole('button', { name: 'Open menu' });
  const menu = page.locator('#menu');

  await trigger.press('Enter'); // open it the way a keyboard user would
  await expect(menu).toBeVisible();

  // move focus off the trigger first
  await page.getByRole('link', { name: 'Item A' }).focus();
  await page.keyboard.press('Escape');

  await expect(menu).toBeHidden();
  await expect(trigger).toBeFocused();
});

Run every model-drafted test against the broken version and watch it fail first.

Gaps in this flow

It doesn't test zoom and reflow, motion or cognitive load. Ten minutes with NVDA also won't tell you how someone who uses a screen reader every day experiences the site. Before a launch, pay disabled people to test it.

The EAA question from your PM

This isn't legal advice. Each point below links to its source.

• The European Accessibility Act applies from 28 June 2025. E-commerce services are in scope, defined as services provided "at the individual request of a consumer with a view to concluding a consumer contract" (Article 3(30)).
• Microenterprises providing services are exempt (Article 4(5)).
• Products and services that meet harmonised standards cited in the Official Journal are presumed to conform (Article 15). EN 301 549 V4.1.1, published by ETSI on 2 September 2026, is based on WCAG 2.2 A and AA, and as of mid-September 2026 it had not been cited there. If you're building or rebuilding now, aim for WCAG 2.2 AA.
• For evidence, keep scan results, manual test notes and known issues with a fix plan.
• On effort: steps 1 and 2 scale with a script. Steps 3 and 4 are manual work per template and per state, so list both (home, product, cart, checkout; menu open, modal, form with errors) before anyone estimates.

Where I'd stop

A verdict on whether a site is accessible, WCAG conformant or EAA compliant stays with a person. So does the call on focus order and on what a screen reader announces. Two smaller things are easy to let slide: closing a violation as a false positive before I've looked at the element, and alt text for an image the model hasn't seen on the page.

If you've got a triage prompt that holds up better than this one, post it in the comments.

AI-assisted. Reviewed and edited by me.