DEV Community: GatekeeperQA

Your accessibility check failed in CI. Keep the evidence before fixing it.

GatekeeperQA — Tue, 20 Oct 2026 17:00:58 +0000

A red accessibility check tells you that a gate failed. The useful next questions are: which element triggered it, what page state was tested, and what evidence will you compare after the change?

The workflow below keeps the original report, generates a separate retest report, and records manual observations alongside the automated result.

Disclosure: I maintain GatekeeperQA, the tool used here. The local CLI runs without a hosted account. Hosted storage and review features are separate, with plan and retention limits.

Keep two reports, not one overwritten folder

With Node.js 24 and an existing package.json, install the pinned CLI:

npm install --save-dev --save-exact @gatekeeperqa/cli@0.3.0
npx --no-install playwright install chromium

Create gatekeeperqa.json:

{
  "urls": ["http://127.0.0.1:3000/"],
  "failOn": ["critical", "serious"],
  "warnOn": ["moderate", "minor", "unknown"],
  "timeoutMs": 30000
}

Start your app and wait until it is ready, then scan:

npx --no-install gatekeeperqa --config gatekeeperqa.json --output reports/before

After implementing your change, run the same configuration against the updated app:

npx --no-install gatekeeperqa --config gatekeeperqa.json --output reports/after

Use fresh directories for each attempt. A setup failure must not leave you interpreting an older report as the latest result.

Open each directory's report.html. Completed scans also produce JSON, Markdown and JUnit XML. The CLI does not directly export a PDF.

This configuration checks an explicitly listed page in its initially loaded state. It does not crawl the site or exercise login, dialogs, checkout, or other interactions. Use only targets you own or have permission to test.

In GitHub Actions, save evidence before enforcing the gate

If the scan exits nonzero, a normal subsequent step can be skipped. Separate evidence collection from the step that marks the job failed.

GatekeeperQA's complete GitHub Actions setup installs the published package, captures the scan exit code, uploads available report artifacts, and then enforces that result.

The important sequence is:

Run the scan and retain its exit code.
Upload the evidence even when the quality gate fails.
Fail the job when the captured result requires it.

Do not turn a failed scan into a successful check just to keep the artifacts. Setup failures may have no report; inspect the logs in that case.

In this CLI version, PASS and WARN exit with 0, FAIL with 1, and setup/configuration errors with 2. WARN still needs review. A gate result is not a compliance certificate.

Also check what the workflow actually targets. Scanning production does not test code that exists only in a pull request. For PR validation, run the checked-out app or scan that PR's deployed preview after it is ready.

Retest the same element and state

Consider an icon-only button:

<!-- Before: no accessible name -->
<button type="button" id="checkout">
  <svg aria-hidden="true" focusable="false"><!-- icon --></svg>
</button>

<!-- After: visible text names the action -->
<button type="button" id="checkout">
  <svg aria-hidden="true" focusable="false"><!-- icon --></svg>
  Continue to checkout
</button>

This is an illustrative fix, not a customer assessment. A subsequent scan may stop reporting the naming issue, but that alone does not verify keyboard activation, focus behavior, or the resulting checkout state.

Before closing the ticket, confirm that the button still exists and was included in the retest. Record the builds, URL, state, engine version and configuration. If those changed, explain the difference.

Write “no longer detected in this scan” when that is what you established. Record keyboard and screen-reader observations separately; leave checks you did not perform marked as not tested.

Give the next reviewer a reproducible handoff

Original report and finding:
Original build:
Updated build:
Page and starting state:
Element location and rule:
Automated retest result:
Keyboard observations:
Screen reader/browser and observations:
Checks not performed:
Reviewer and date:
Remaining issues:

GatekeeperQA's hosted review workflow can keep saved notes and prepare an editable GitHub issue draft. Its private finding links open the specific element in a retained report. Those links require the report owner's sign-in; pasting one into a team ticket does not grant teammates access. Include appropriately redacted evidence that your intended reviewer can actually read.

You can also keep this information in your existing tracker and retain the local reports. The workflow is useful independently of hosted storage.

For a recorded walkthrough, see the developer guide. For a quick look without registration, open the interactive sample.

Where does your team's accessibility handoff break down: locating the element, reproducing the state, or deciding what has really been verified?

Prepared with AI assistance and checked against the project's CLI documentation and reporting implementation. Automated testing covers only part of accessibility; this is not a completed accessibility assessment.

How to retest an accessibility fix and write useful evidence

GatekeeperQA — Wed, 30 Sep 2026 16:35:05 +0000

A developer fixes an unnamed button. The next scan no longer reports it. Before closing the ticket, what should you verify?

Repeat the original check, test the interaction, and record the evidence. This fictional checkout example shows a focused retest workflow you can use in your existing issue tracker.

Start with one clear fix

An icon-only button has no accessible name. Adding visible text makes its purpose available to users:

<!-- Before -->
<button type="button" id="checkout">
  <svg aria-hidden="true" focusable="false"><!-- Cart icon --></svg>
</button>

<!-- After -->
<button type="button" id="checkout">
  <svg aria-hidden="true" focusable="false"><!-- Cart icon --></svg>
  Continue to checkout
</button>

The label addresses the naming problem. The application still needs to handle activation correctly. W3C's button pattern describes naming, keyboard activation and focus behavior.

1 Repeat the automated check

Open the same page and state, such as a cart containing an item. Record the original and updated build identifiers. Keep the engine version and scan settings comparable, or document what changed.

Confirm the retest completed and included the affected button. An empty cart, missing component or failed scan cannot verify the fix.

A useful result is specific: “The accessible-name finding was not detected for the checkout button in the populated-cart state.” Use that wording only when your evidence supports it. Keep both reports so another reviewer can compare them.

2 Test the interaction

For this button, check:

Keyboard: Reach it with Tab and Shift+Tab. Check visible focus and navigation order. Test Enter and Space separately from the same starting state; each should perform the intended action once.
Screen reader: Check the announced name and button role. Record the browser, screen reader and versions used.
Resulting state: Follow the checkout action. If it opens a dialog, check focus inside it and after closing it. Exercise applicable loading and error states, including how the user recovers.

These checks verify a specific interaction. They do not establish whole-site accessibility. W3C explains why automated tools need human evaluation.

3 Keep the evidence with the ticket

Record each check as passed, failed or not tested according to what you actually verified. These statuses describe the page assessment. Keep automated results and manual observations separate.

Use this compact template:

Page, component and starting state:
Original issue and evidence:
Expected behavior:
Fix build or commit:
Retest date and reviewer:
Browser and assistive technology versions:
Scan engine, settings and result:
Keyboard and screen-reader observations:
Other states checked and coverage gaps:
Remaining issues and linked tickets:
Decision against acceptance criteria:

If the label is fixed but focus remains broken, record that remaining issue and assign it. Close the ticket against its acceptance criteria, with enough detail for another tester to repeat the check. Redact personal information and secrets before sharing evidence.

Put the workflow to use

I maintain GatekeeperQA. GatekeeperQA is ready to use for accessibility reporting and retest tracking, with findings, reviewer notes, ticket copying and retest comparisons.

Explore the free interactive sample without signing up, or scan a public page you own or have permission to test. Usage and shared-capacity limits apply.

Try it in your workflow. Your feedback helps shape what we build next.

Disclosure: Drafted with AI assistance. This fictional example is not a client assessment.