A finding disappears from the next scan. Can your report call it fixed?
Not yet. The element could have moved, the page could be incomplete, or the rule could now require manual review.
We build GatekeeperQA. While auditing its retest workflow after community feedback, we found a mismatch: individual-finding retests correctly reported uncertainty when a later scan had an unresolved check for the same rule, but the bulk comparison could count that finding as “no longer detected.” We corrected the bulk result and added regression coverage.
One example, three different conclusions
Consider an earlier label violation on a form field. These are illustrative scenarios, not customer results:
| Later scan | Safe conclusion | Next step |
|---|---|---|
| The same finding is present | Still detected | Inspect the fix and retest |
| The rule has an unresolved manual check | Uncertain / not verified | Review the affected control manually |
| No matching finding, with comparable completed coverage | No longer detected | Verify that the same control remains and works |
An unreviewed check must stay unreviewed. A missing page result or execution error cannot establish a passing comparison. “Fixed” needs evidence appropriate to the issue, not just an empty list.
Match the finding, not its row number
Rows can move between reports. One approach is an exact-location identifier built from the page URL, rule and selector structure. It stays consistent when rows are reordered.
That is not a permanent DOM identity: a changed selector, different URL or replaced element still needs investigation. Preserve the original report references instead of assuming two similar-looking rows describe the same control.
Give the next reviewer enough context
For a useful handoff, record:
- The earlier and later report IDs and scan dates.
- The tested page and finding location.
- The automated comparison result, separate from human review.
- What a person tested, which input method they used, and what remains untested.
Store structured report references and dates with attached retest evidence, and carry them into exports. Do not infer dates or version information that older text-only notes never recorded.
Regression tests should cover row reordering, unresolved checks and incomplete page coverage. Passing those tests is software regression evidence, not proof of accessibility conformance. Keyboard, screen-reader and other relevant manual checks remain necessary.
A regression case worth keeping
Given an earlier label violation and a later unresolved label check, assert that the bulk comparison reports uncertainty instead of counting the finding as no longer detected. Test the individual result and the bulk summary against the same fixture so they cannot quietly disagree.
Disclosure: this article was generated with AI and checked against the project implementation and test evidence. The example scenarios are illustrative, not firsthand user-testing results.
Top comments (1)
You did a great job explaining how the Dominant Eye Shader gradually reduces its alpha from 0.6 to 0.2 and how that translates into a clearer view for users. One practical improvement would be to expose that alpha value as an in-app slider that saves the user's preference (e.g., via PlayerPrefs or a small JSON file). Pair the slider with a simple contrast check that warns if the resulting image falls below WCAG 2.1 AA standards, and run a quick screen-reader test on the UI controls to ensure they're fully reachable.
If you ever need a fast sanity check, something like Scanly I use for quick 60-second accessibility audits can highlight missing ARIA labels or color-contrast issues before you ship.
Have you considered adding a brief onboarding tutorial that explains the shader controls for first-time users? Disclosure: I'm affiliated with Scanly.