DEV Community

Cover image for axe passes but the component is still unusable: five bugs automated tools can't catch
Duncan Faulkner
Duncan Faulkner

Posted on Edited on Originally published at ngbracket.com

axe passes but the component is still unusable: five bugs automated tools can't catch

A green accessibility report is easy to trust. The CI check passes, Lighthouse says 100 and the axe panel shows no violations. You ship it, and a person using a screen reader reaches your component and can't use it.

I've shipped components like that more than once. What I learned is that automated accessibility testing checks whether your HTML is well-formed. It doesn't check whether your interface is usable. Users get stuck in the gap between those two questions.

axe is very good, and you should run it in CI if you don't already. It works by inspecting a snapshot of the DOM against a set of rules. It can see that an attribute is present. It can't see whether the attribute is true, whether the order of things makes sense, or whether anything happens when you press a key. It catches roughly a third of WCAG issues. A person has to check the rest.

Here are five components that all pass axe.

1. The modal that drops your focus

A dialog opens, you confirm, it closes. axe has no complaints: role="dialog", aria-modal, a labelled heading, focus trapped while it's open. All correct.

confirm() {
  this.save();
  this.open.set(false); // ✅ AXE passes. ❌ focus just fell to <body>.
}
Enter fullscreen mode Exit fullscreen mode

A keyboard user sees the dialog close and their focus land on <body>, at the top of the page. They've lost their place and have to Tab back through the whole UI. axe can't catch this. Where focus goes after a state change is a question about time, and the DOM snapshot doesn't hold it.

The fix is to remember the trigger and return focus to it:

private trigger: HTMLElement | null = null;

openDialog() {
  this.trigger = this.doc.activeElement as HTMLElement;
  this.open.set(true);
}
close() {
  this.open.set(false);
  this.trigger?.focus(); // put them back where they were
}
Enter fullscreen mode Exit fullscreen mode

2. The error you can only see if you can see

A form field fails validation and a red border appears. axe checks that the label is present and the border colour has enough contrast. Both are fine.

But colour is the only signal that something went wrong. Some colour-blind users won't see it. A screen-reader user hears nothing, because the message was never tied to the field or announced:

<!-- passes AXE, communicates nothing -->
<input formControlName="email" class="error">
<p class="error-text">Enter a valid email</p>
Enter fullscreen mode Exit fullscreen mode

Nothing connects that <p> to the input, and nothing tells assistive technology it appeared. The fix is to make the error programmatic:

<input formControlName="email"
       [attr.aria-invalid]="showError()"
       [attr.aria-describedby]="showError() ? 'email-error' : null">
@if (showError()) {
  <p id="email-error" role="alert">Enter a valid email.</p>
}
Enter fullscreen mode Exit fullscreen mode

aria-invalid says the field has an error, aria-describedby ties the explanation to it, and role="alert" announces it when it renders. axe won't ask for any of the three. It only knows the label exists.

3. The toggle that looks operable but does nothing

The ARIA here is correct:

@Component({
  host: {
    'role': 'switch',
    '[attr.aria-checked]': 'checked()',
    'tabindex': '0',
    // …no keyboard handler.
  },
})
Enter fullscreen mode Exit fullscreen mode

axe sees a valid switch with a valid aria-checked state and a tabindex. It can't know that pressing Space does nothing. Finding that out means running the control, and axe only reads it. A mouse user can toggle it. A keyboard user can focus it and then can't change it.

host: {
  'role': 'switch',
  '[attr.aria-checked]': 'checked()',
  'tabindex': '0',
  '(keydown.space)': 'toggle(); $event.preventDefault()',
  '(keydown.enter)': 'toggle()',
}
Enter fullscreen mode Exit fullscreen mode

A better option is a native <button> with aria-pressed for the state. A native button handles Space and Enter itself, so it can't have this bug.

4. The live region that never speaks

You add an aria-live region that says "12 results" after filtering, like this:

@if (announcement()) {
  <p aria-live="polite">{{ announcement() }}</p>
}
Enter fullscreen mode Exit fullscreen mode

axe sees a valid live region. But screen readers only announce changes to a live region that was already in the DOM when the change happened. Here the @if adds the element and its text at the same moment, so nothing is announced. The region has to exist empty first and then receive the text:

<p class="sr-only" aria-live="polite">{{ announcement() }}</p>
Enter fullscreen mode Exit fullscreen mode

Or use the CDK's LiveAnnouncer, which keeps a persistent region for you:

private announcer = inject(LiveAnnouncer);
this.announcer.announce(`${count} results`, 'polite');
Enter fullscreen mode Exit fullscreen mode

This is my favourite example, because the broken version looks more correct than the one that works. A tool can't tell them apart. A screen reader tells you straight away.

5. The chart a screen reader can't read

You render an SVG line chart. Contrast meets AA, there's a <title>, and axe passes. To a screen-reader user it's an empty image. SVG internals aren't exposed as data, and no automated rule can say "this picture holds information that appears nowhere else on the page".

The fix is to give the same data in a form assistive technology can read, such as a visually hidden table behind the chart:

<figure>
  <svg aria-hidden="true">…</svg>
  <table class="sr-only">
    <caption>Monthly revenue, Jan–Dec</caption>
    <tr><th>Month</th><th>Revenue</th></tr>
    <!-- one row per point -->
  </table>
</figure>
Enter fullscreen mode Exit fullscreen mode

This is the biggest blind spot in automated testing. It's also why charts are among the hardest things to make accessible on the front end.

What to test by hand

In all five cases axe was right: the HTML was well-formed. Each bug was in something a snapshot can't see: order, interaction, timing or meaning.

So keep axe in CI. It's a good regression check for the third of issues it covers, and it's free. Then add two manual checks, which take a couple of minutes:

  1. Put the mouse down and Tab through the page. Can you reach and operate everything? Does focus go somewhere sensible after each action?
  2. Turn on a screen reader, either VoiceOver (⌘+F5) or NVDA, and listen. Does what you hear match what's on screen?

Those two checks cover a lot of what axe can't. If you only run the automated check, you know your HTML is tidy, but not whether people can use what you built.


I built NgBracket because of this gap. It's a set of Angular component packs built to support WCAG 2.2 AA. Every documented component is checked with axe in CI, I test each new component with the keyboard, and a suite of automated NVDA tests runs against the demo app. The conformance report lists how each criterion is tested and the known limitations. You can start with the free packs without a card.

Top comments (1)

Some comments may only be visible to logged-in visitors. Sign in to view all comments.