Text at 1:1 contrast is not an axe violation. It is incomplete.

I shipped a button whose label was invisible. Same colour as its own background — 1:1 contrast, no readable text at all. It was only broken in dark mode, and my CI was green the entire time.

It was green for two reasons. I expected the first one. The second one is why I am writing this.

Reason one: the headless browser boots in light mode

axe evaluates the colour scheme that is actually rendered. It has no way to reason about a palette that was never painted. My pipeline loaded the page once, in whatever state Chromium starts in, and that state is prefers-color-scheme: light.

That part is well known, and the fix is one line:

const context = await browser.newContext({ colorScheme: "dark" });

So I did that, re-ran, and the run was still green.

Reason two: axe does not call this a violation

Here is the probe I wrote when I stopped believing the output. Same page, same axe config, two colour schemes:

=== scheme: light ===
{ matchesDark: false,
  color: 'rgb(255, 255, 255)',
  background: 'rgb(28, 93, 63)' }
violations: []
incomplete: []

=== scheme: dark ===
{ matchesDark: true,
  color: 'rgb(108, 196, 154)',
  background: 'rgb(108, 196, 154)' }
violations: []
incomplete: [ 'color-contrast x1' ]

In dark mode the text and its background are the same value. rgb(108, 196, 154) on rgb(108, 196, 154). You cannot read it, because there is nothing to read.

and violations is empty.

It lands in incomplete.

axe is not wrong

This is the part that took me a while to accept. axe declines to call it a violation because it genuinely cannot tell the difference between:

• text that is invisible because someone broke the palette, and
• text that is invisible on purpose — a visually-hidden label, a fade-in that has not started, a print-only string, a decorative element.

Matching foreground to background is a legitimate hiding technique. axe reports what it can prove and hands the rest to a human. That is the right call for a linter.

It is the wrong call for my pipeline, because my pipeline asserts on violations and throws incomplete away. Almost every pipeline does. Look at your own reporter and count how many of them you have ever read.

So the failure mode is asymmetric in the worst direction: the more severe the contrast defect, the more likely it is to be classified as needing review rather than failing. A 4.3:1 ratio is a confident violation. A 1:1 ratio is a question.

What I changed

Two things, and the first one matters more than the second.

Collect incomplete, and label it distinctly. Not "treat it as a failure" — that will bury you. Label it, surface it, and look at the ones that are new.

Also: do not set resultTypes: ["violations"]. It looks like a harmless optimisation. It makes axe return a single representative node for every other bucket, so the incomplete results get silently truncated before you ever see them.

Diff across states instead of counting. A raw count of findings is a chore that nobody triages. "These 3 findings exist in dark mode and not in your baseline" is a bug report with a cause attached.

That second idea turned into a small tool: a11y-matrix. It runs axe across dark mode, reduced motion, forced colors, mobile and the 320px reflow width, changing exactly one variable at a time, and reports only what each state uniquely breaks. MIT, and it runs from npx github:henriqueyuri00/a11y-matrix <url>.

One more thing it taught me

I pointed it at my own sales page. The baseline was clean. At 320px it reported six findings: a comparison table overflowing and covering its own third column.

So I fixed it the recommended way — overflow-x: auto, tabindex="0", role="region", an aria-label — and re-ran.

Still six findings. Identical message.

Because axe emits the same string for "covered by an overlay" and "scrolled out of view inside a scroll container". The first is a defect. The second is the sanctioned way to present a wide data table at the reflow width, and it is what the spec asks you to do.

My own tool was failing a page for being fixed correctly. That is how a check earns its way out of a pipeline within a week — so it now walks the DOM, asks whether the element sits inside an ancestor that actually scrolls horizontally and that a keyboard can reach, and suppresses the finding while printing the count and the reason. Never silently.

And since that removed the only thing watching reflow, it measures reflow directly instead: document scrollWidth against viewport width. axe has no rule for WCAG 1.4.10 at all. Stating the requirement beats inferring it from a contrast side effect.

---

If your accessibility pipeline is green, that is worth exactly as much as the states you rendered and the buckets you read.

Comments

[Tobias]

"The more severe the contrast defect, the more likely it is to be classified as needing review rather than failing" is the sentence, and it generalises well past axe.

I hit the identical shape in a different validator. Schematron reports two result kinds: a failed assertion, and a successful report, which fires when its condition is true. Our parser read the first and silently discarded the second, exactly the way a pipeline asserts on violations and throws incomplete away. Whole bucket gone, and the output looked clean.

The second-order trap is the one I'd flag, because it's the obvious fix and it's wrong. When we started reading that bucket, we defaulted its severity to error. That immediately failed the standards body's own reference sample, which legitimately carries about eleven of those notices in its smallest profile and several hundred in its largest. The bucket exists because the tool honestly cannot decide, so promoting it to failure just relocates the lie.

Which is to say your first change is the right one and the ordering matters: collect it, label it distinctly, look at what's new. Not treat it as a failure. We learned that in the other order.

Diff-across-states is the good idea here regardless. A count is a chore. "This state uniquely breaks these three" is a bug report with a cause attached.