My accessibility scanner cried wolf on a real site, so I fixed the scanner and put it in CI

#a11y #webdev #githubactions #compliance

The European Accessibility Act has been enforceable for over a year now, and the tooling situation for developers is still awkward: every scanner flags everything, so teams learn to ignore the red — which is worse than having no scanner at all.

I built a WCAG 2.2 audit engine that runs as a GitHub Action. The design rule is "inform aggressively, block conservatively": the build fails only on determinate WCAG Level-A defects. Warnings, heuristics, and things automation genuinely can't prove still get reported — they just don't hold your deploy hostage.

Then I ran it on a real production site, and my own scanner was the thing that was wrong.

The finding that turned out to be my bug

The target: a live veterinary-practice PWA (vetcarepros.com) — real users, real stakes. The first audit came back with 2 Level-A errors: elements unreachable by keyboard and no visible focus indicator, both anchored on the same component — the site's reCAPTCHA badge.

Serious if true. Keyboard traps are exactly the kind of thing the EAA cares about.

Except it wasn't true. The badge is a presentational utility: tabindex="-1", role="presentation", deliberately removed from the tab order by design — the same pattern Radix and Headless UI use for form shims. My probe was correctly detecting the element and correctly observing it couldn't be tabbed to. What it couldn't do was tell "hidden on purpose and declared so" apart from "broken."

That distinction — between a finding and a false positive — is the whole game. A scanner that can't make it trains developers to dismiss every result it produces.

What the fix actually looks like

The detector now suppresses only when both signals are present:

the element is declared non-interactive (tabindex="-1", role="presentation|none"), and
it has trusted-widget provenance (a known embed source like reCAPTCHA/App Check, or a recognized wrapper).

An element that hides itself without declaring it still fails as a conclusive Level-A error. A visible embed missing either signal still fails. You can't escape a finding by slapping tabindex="-1" on it — the suppression needs the full story.

And here's the part I'm more proud of: the suppressed findings aren't deleted. They're reported as info with disposition: false_positive — disclosed in the findings report, the CSV export, and the VPAT remarks as "observation(s) dispositioned as not-applicable." Hiding them would be the same lie as over-flagging, just quieter.

Same site, re-scanned: 0 errors, 0 warnings, 2 disclosed suppressed findings. That's what "clean" should look like.

The honesty problem nobody talks about

The deeper issue in a11y tooling is grading honesty. Most WCAG criteria can't be fully proven by automation — "is this title meaningful?" isn't a regex. The engine's answer is boring but strict: if the checks can't prove conformance, the row says Not Evaluated and names the manual task you'd need to do. It never claims Supports because nothing went wrong; it claims it only when conclusive checks ran clean on counted elements.

A compliance report that hedges honestly is worth more than one that's confidently wrong — because the former is evidence and the latter is a liability.

What you get in CI

Build fails on determinate Level-A defects only (fail-on: 'level-a', or 'any-error', or 'never')
PR comment + step summary with the findings table
Same audit run powers a downloadable VPAT/ACR and findings CSV in the portal
Optional Merkle hash-chain verification of your audit trail (verify-chain: 'true')
EU data residency — sovereign partitions, geo-routed
Perimeter-only: it scans the deployed URL. Your source code never leaves your repo — and the action source is public so you can verify that yourself.

Try it

Free during the developer beta — no card, 90-day keys:

curl -X POST https://api.compliance-audit.eu/v1/demo/keys \
  -H 'Content-Type: application/json' \
  -d '{"companyName": "your-project"}'

Then:

- uses: Terrygilligan/compliance-audit-verify-action@v1
  with:
    api-key: ${{ secrets.COMPLIANCE_API_KEY }}
    tenant-id: ${{ secrets.COMPLIANCE_TENANT_ID }}
    target-url: 'https://your-preview-deploy.example.com'
    fail-on: 'level-a'

Action repo: Terrygilligan/compliance-audit-verify-action (MIT). Zero-dependency — it's a 300-line Node script, read it before you run it. Portal with the live audit sandbox (no key needed): compliance-audit.eu.

Bugs and false positives — especially false positives — go in the repo's issues. If your site trips a wrong finding, that's a bug in my detector, not your markup, and I want to see it.

DEV Community