I build Canli Capital: free, open-source tools that check whether a backtest is real (deflated Sharpe, overfitting and data-snooping tests), plus SEC fundamentals for AI agents over MCP.
The closed loop ("fix every violation, then scan again") has a failure mode that the violation count can't see: the cheapest way to make img-alt or link-name go to zero is to delete the element, or to write alt="image". A re-scan reports a clean page either way. It would help to return something the agent can't satisfy by deletion, for example the element count per rule before and after (so "3 images checked, 0 now" reads differently from "3 images, all with alt"), and a review item for alt text that equals the filename or a stock word.
Same thing for rulesPassed: 28 in your sample: with two violations and one review out of 31, most of those passes are rules that found nothing to test, not rules that tested something and passed. For scan_html on a fragment it matters more, since contrast, heading order, landmark and skip-link rules depend on the page around the component. A separate notApplicable bucket (and a note when a fragment was scanned without the app's stylesheet) would stop the agent from reading "28 passed" as evidence about the whole page.
We build A11yScope, a free Chrome extension that checks web pages against WCAG 2.2 AA. Writing about accessibility, the European Accessibility Act, and what automated checks miss.
@arhancanli You are right on both counts, and the first one was a real hole in the "fix it, then scan again" loop. Shipped today as a11yscope-mcp 0.2.0 (engine commit 6480ca5):
Every rule now declares what it looks at, and reports checked (how many such elements were in the DOM: every <img> for img-alt, every link for link-name, 1 for page-level rules) plus a status: failed, review, passed or inapplicable. A rule with checked: 0 and no findings is inapplicable, never passed.
The summary splits rulesPassed from rulesNotApplicable. Your reading of my sample was correct: re-running it, <main><h1>Hi</h1><img><a href="/x"></a></main> gives 1 passed and 26 not applicable, not 28 passed. The server instructions now tell the agent to compare checked before and after, so "3 images checked, 0 now" is visible as deletion.
scan_html adds notes when the markup is a fragment (page-level checks then describe the fragment, not the page it will live in) and when there is no stylesheet at all (contrast, focus and target-size then reflect browser defaults). Both scan tools also report iframesNotEntered.
On alt="image" and alt equal to the file name: that was already covered by a separate rule, img-alt-redundant, which flags an alt that ends in an image extension or is a filler word (image, img, photo, picture, graphic, icon, logo, spacer). It is a violation, not a review item, so an agent that "fixes" img-alt by writing alt="image" trades one violation for another. What it cannot judge is whether alt="Team photo" is honest, and that stays with the human.
One caveat I want to be precise about: checked counts elements present in the DOM, before the rule's own exclusions (hidden, aria-hidden, decorative). So a hidden <img> with no alt shows as checked 1 / passed, because the rule deliberately does not report what nobody can reach. The count answers "did the elements survive", not "did every one get judged".
I build Canli Capital: free, open-source tools that check whether a backtest is real (deflated Sharpe, overfitting and data-snooping tests), plus SEC fundamentals for AI agents over MCP.
The caveat you flag points at the next loophole, I think: if checked counts elements before the exclusions, an agent can clear a failing img-alt by adding aria-hidden="true" (or role="presentation", or hidden) to the image. checked stays 1, the status flips to passed, and the before/after comparison shows nothing, because the element survived and was never judged.
A cheap closer is a third number per rule, excluded (how many of the checked elements the rule skipped as hidden or decorative), with the same compare-before-and-after instruction. A scan that goes from checked 3 / excluded 0 to checked 3 / excluded 3 reads as "everything was hidden", which is the case worth stopping on. Genuine decorative images exist, so I would report it as review, not failed.
We build A11yScope, a free Chrome extension that checks web pages against WCAG 2.2 AA. Writing about accessibility, the European Accessibility Act, and what automated checks miss.
@arhancanli Yes, that is the next loophole, and the cheap closer is the right one. Committed to main today, goes out as 0.2.1:
Every element-level rule now reports excluded next to checked: how many of the candidates it skipped because they are hidden from assistive technology (aria-hidden, display:none, hidden) or marked decorative (role="presentation" / "none"). Your case, <img src="a.png" aria-hidden="true"> with no alt, now comes back as img-alt: passed, checked 1, excluded 1 instead of a silent pass, and the test suite pins that.
The server instructions tell the agent to compare checked and excluded before and after, and to treat a rise in excluded as a question for a human rather than a fix, for exactly the reason you give: decorative images are real, so the tool cannot call it a failure, but it can refuse to call it progress.
One rule opts out: aria-hidden-focusable, whose candidates are aria-hidden by definition, so counting them as excluded would say nothing.
What this still does not catch is an agent that keeps the element visible and writes a plausible lie in the alt. Nothing automated does, and the result says so in every response.
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
The closed loop ("fix every violation, then scan again") has a failure mode that the violation count can't see: the cheapest way to make
img-altorlink-namego to zero is to delete the element, or to writealt="image". A re-scan reports a clean page either way. It would help to return something the agent can't satisfy by deletion, for example the element count per rule before and after (so "3 images checked, 0 now" reads differently from "3 images, all with alt"), and areviewitem for alt text that equals the filename or a stock word.Same thing for
rulesPassed: 28in your sample: with two violations and one review out of 31, most of those passes are rules that found nothing to test, not rules that tested something and passed. Forscan_htmlon a fragment it matters more, since contrast, heading order, landmark and skip-link rules depend on the page around the component. A separatenotApplicablebucket (and a note when a fragment was scanned without the app's stylesheet) would stop the agent from reading "28 passed" as evidence about the whole page.@arhancanli You are right on both counts, and the first one was a real hole in the "fix it, then scan again" loop. Shipped today as
a11yscope-mcp0.2.0 (engine commit 6480ca5):checked(how many such elements were in the DOM: every<img>forimg-alt, every link forlink-name, 1 for page-level rules) plus astatus:failed,review,passedorinapplicable. A rule withchecked: 0and no findings isinapplicable, neverpassed.rulesPassedfromrulesNotApplicable. Your reading of my sample was correct: re-running it,<main><h1>Hi</h1><img><a href="/x"></a></main>gives 1 passed and 26 not applicable, not 28 passed. The server instructions now tell the agent to comparecheckedbefore and after, so "3 images checked, 0 now" is visible as deletion.scan_htmladdsnoteswhen the markup is a fragment (page-level checks then describe the fragment, not the page it will live in) and when there is no stylesheet at all (contrast, focus and target-size then reflect browser defaults). Both scan tools also reportiframesNotEntered.On
alt="image"and alt equal to the file name: that was already covered by a separate rule,img-alt-redundant, which flags an alt that ends in an image extension or is a filler word (image, img, photo, picture, graphic, icon, logo, spacer). It is a violation, not a review item, so an agent that "fixes"img-altby writingalt="image"trades one violation for another. What it cannot judge is whetheralt="Team photo"is honest, and that stays with the human.One caveat I want to be precise about:
checkedcounts elements present in the DOM, before the rule's own exclusions (hidden,aria-hidden, decorative). So a hidden<img>with no alt shows as checked 1 / passed, because the rule deliberately does not report what nobody can reach. The count answers "did the elements survive", not "did every one get judged".The caveat you flag points at the next loophole, I think: if
checkedcounts elements before the exclusions, an agent can clear a failingimg-altby addingaria-hidden="true"(orrole="presentation", orhidden) to the image.checkedstays 1, the status flips to passed, and the before/after comparison shows nothing, because the element survived and was never judged.A cheap closer is a third number per rule,
excluded(how many of the checked elements the rule skipped as hidden or decorative), with the same compare-before-and-after instruction. A scan that goes from checked 3 / excluded 0 to checked 3 / excluded 3 reads as "everything was hidden", which is the case worth stopping on. Genuine decorative images exist, so I would report it as review, not failed.@arhancanli Yes, that is the next loophole, and the cheap closer is the right one. Committed to main today, goes out as 0.2.1:
excludednext tochecked: how many of the candidates it skipped because they are hidden from assistive technology (aria-hidden,display:none,hidden) or marked decorative (role="presentation"/"none"). Your case,<img src="a.png" aria-hidden="true">with no alt, now comes back asimg-alt: passed, checked 1, excluded 1instead of a silent pass, and the test suite pins that.checkedandexcludedbefore and after, and to treat a rise inexcludedas a question for a human rather than a fix, for exactly the reason you give: decorative images are real, so the tool cannot call it a failure, but it can refuse to call it progress.aria-hidden-focusable, whose candidates are aria-hidden by definition, so counting them as excluded would say nothing.What this still does not catch is an agent that keeps the element visible and writes a plausible lie in the alt. Nothing automated does, and the result says so in every response.