DEV Community: Mohsen Vaziri

Before the 304-site study, I scanned 57 agency front pages. Here is what one scan can and can't tell you.

Mohsen Vaziri — Thu, 05 Nov 2026 22:08:18 +0000

Earlier I posted what I found when I scanned 304 web agency front pages. That study didn't start at 304. It started in September with 57 sites, and the small one is where I worked out the method. It's also where I had to think hardest about what a single automated scan can honestly claim, so I want to share it too.

The setup: on 16 and 25 September 2026 I scanned the front pages of 57 web design and development agencies with Sitejar, the scanner I build. One read-only scan per site, front page only. No crawl, no login, no form submissions, and sites whose robots rules block automated visits were skipped. Six areas were checked: accessibility, security, privacy, deceptive design, technical quality and SEO. No site is named, and no figure describes fewer than five sites.

The headline numbers

Accessibility: all 57 fell short of full marks. Worst score 56, median 78, best 96.
Security: 50 of 57 (87.7%) fell short. Worst 48, median 74.
Privacy: 48 of 57 (84.2%) fell short. The lowest score in the whole study, 26, was here.
Technical quality: 43 of 57 (75.4%) fell short, median 94.
SEO: only 7 of 57 (12.3%) fell short, and none by more than five points.
Deceptive design: 0 of 57. More on that below.

The median site had 14 findings (mean 13.7).

The findings, and how sure the scanner was

Finding	Sites	Share	Confirmed
Low text contrast	45	79%	43
Missing Content-Security-Policy header	40	70%	40
Link without a discernible name	23	40%	23
No link to an accessibility statement	20	35%	20
Content outside any landmark region	19	33%	19
Missing Strict-Transport-Security header	18	32%	18
Third-party cookie set before consent	16	28%	16
Form field missing a label	14	25%	14
Button or control without an accessible name	10	18%	10
Tracking script loading before consent	9	16%	9
Scrollable region not keyboard-focusable	7	12%	7
Sensitive form data sent with GET	7	12%	7
Missing alt text	6	11%	6
Skipped heading level	5	9%	5
Duplicate landmark labels	5	9%	5

Twenty more findings showed up on fewer than five sites each. I left them out, because with 57 sites a finding on two or three of them is close to pointing at who they are.

The Confirmed column is the part I care about

Every finding gets one of three labels. Confirmed means the rule is deterministic and the failure is certain from the page itself, like a header that isn't in the response or an input with no label in the DOM. Likely means the rule fired but a person could reasonably disagree. Needs review means the scanner can't decide and wants a human.

Fourteen of the fifteen findings were Confirmed on every site that had them. The one exception is contrast: 45 sites, 43 Confirmed. On the other two, the text sat over an image or a gradient, so there was no single background colour to measure against. The scanner said Likely and asked for a person to look. I'd much rather a tool says "not sure" there than quietly counts it as a fail, or skips it.

Also worth saying: a Confirmed finding means a barrier exists on the page as it was served. It doesn't tell you how many people it affects, whether the site breaks any law, or how it compares with a competitor.

Contrast and CSP are two very different failures

Low contrast was the most common finding, on 79% of sites. It's almost never carelessness. It's usually one shared colour token, a light grey picked for captions or a footer, used everywhere at once. Fix the token and you fix it everywhere.

A missing CSP is a different thing. It's not a design call that went a bit wrong, it's a control that was never set up. 70% of these front pages sent no Content-Security-Policy at all, and 32% had no HSTS either. Both are server or CDN config, not page code. My guess from this is that headers fall into the gap between the people who write the pages and the people who run the infrastructure.

About a quarter tracked before consent

A third-party cookie was set before any consent choice on 16 sites (28%), and a tracking script loaded before consent on 9 (16%). Together that's roughly a quarter of agency front pages tracking a visitor before the visitor chose anything. The site owner sees a cookie banner and assumes it's doing its job. The network requests say otherwise.

Why two areas came back clean

Every site scored 100 on deceptive design, and 50 of 57 scored 100 on SEO. It's tempting to read that as "agencies don't use dark patterns and have great SEO". I don't think the data supports that.

Dark patterns live in flows: checkouts, subscription forms, cancellation pages. An agency front page is a portfolio and a contact form. There was very little for those rules to test, so 100 here means nothing was found where there was little to find. SEO is similar but weaker: titles and descriptions are exactly what a professional front page gets right, and a front-page scan doesn't reach the deeper pages where SEO problems pile up.

Looking at all six together, the pattern is pretty clear. The areas a client or a search engine can see (SEO, technical quality) score well. The ones nobody notices until a disabled user, an attacker or a regulator shows up (accessibility, security headers, consent timing) score badly. The things that get checked get fixed.

Compared with the rest of the web

The WebAIM Million 2026 found a detectable WCAG failure on 95.9% of the top million home pages. Here it was 100% of 57. Different tools and different rules, so it's not a direct comparison, and on 57 sites that gap means nothing. What it does support is a smaller claim: agency front pages are not measurably better on accessibility than the web as a whole. Low contrast is also the top failure in WebAIM's data, and missing labels, missing alt text and empty links show up in both.

The 304-site follow-up had enough sites to look at this finding by finding, so if you want the more detailed comparison, it's in the other post.

Limits

57 is small. A finding at 12% (7 of 57) could plausibly be anywhere from about 5% to 23% in the real population. That's exactly why I ran the bigger one.
Not a random sample. It leans toward agencies that are easy to find and allow automated visits.
Front page only. Anything deeper, behind a login or inside a flow isn't in the data.
Automated checks have a ceiling. Deque's coverage report found automated testing caught 57.38% of accessibility issues by volume. So what the scan found is a floor on the real number of barriers, not a full count. A site that scored 96 is not 96% accessible.
It's a snapshot of two days in September 2026.
Scores are Sitejar's own. Don't compare them with Lighthouse or other tools. The findings travel better than the scores.

If you want to compare your own site

Compare findings, not scores. Scan your front page the same way and hold your Confirmed findings against the table above, row by row. And if you only fix two things, fix the two most common ones, which are also among the cheapest: your grey text token, and a CSP header (plus HSTS while you're in the server config).

The full paper

Method, both tables, and a whole section on what this data can't tell you: How Web Agency Front Pages Really Perform: A Six-Dimension Benchmark of 57 Sites.

DOI: 10.5281/zenodo.23986811. The 304-site follow-up: 10.5281/zenodo.23986917.

If you want to check your own front page, Sitejar has a free plan.

We scanned 304 web agency front pages. Here is what failed most.

Mohsen Vaziri — Thu, 05 Nov 2026 18:54:47 +0000

Web agencies sell accessibility, security and privacy work to their clients. So I wanted to know how their own front pages do.

Between 16 September and 23 October 2026, I scanned the front pages of 304 third-party web design and development agencies with Sitejar, the scanner I build. Each site got one read-only scan of its front page, across six areas: accessibility, security, privacy, deceptive design, technical quality and SEO. No site is named, and no published figure describes fewer than five sites.

This extends an earlier 57-site study about fivefold, so most figures now carry an interval of roughly plus or minus five points instead of twelve.

The headline numbers

Accessibility: at most 1 of the 304 front pages had no accessibility finding. 99.7% or more fell short.
Security: 91% fell short of full marks.
Privacy: 85% fell short.
Technical quality: 84% fell short.
SEO: only 12% fell short, and none by more than six points.
Deceptive design: no findings at all. More on why below.

The most common findings

Finding	Share of sites	95% CI
Low text contrast	82%	77.5 to 86.1
Missing Content-Security-Policy header	61%	55.6 to 66.5
Link without a discernible name	39%	33.8 to 44.7
Missing Strict-Transport-Security header	38%	32.9 to 43.7
Content outside any landmark region	36%	31.0 to 41.7
No link to an accessibility statement	29% to 30%	24.8 to 35.0
Third-party cookie set before consent	21% to 22%	17.4 to 26.7
Tracking script loading before consent	21%	17.1 to 26.3
Form field missing a label	20%	15.9 to 24.9

Some shares are given as a range because one site, our own, was removed from the sample after counting. The paper explains exactly how that was handled.

Contrast fails about as often as on the rest of the web

The WebAIM Million 2026 report found low contrast text on 83.9% of the top million home pages. We found it on 82% of agency front pages. The tools and rules differ, so this is not a direct comparison, but it is close enough to notice.

Low contrast is rarely a skill problem. It is usually one design decision: a light grey for captions, placeholders or a footer, defined once as a token and inherited everywhere. Thin type and soft greys look refined on a big, bright monitor and fail the 4.5:1 ratio. The fix is just as systematic: change the token once, at the source.

Where agencies do better

On structural accessibility, agency front pages looked much better than the web as a whole. WebAIM reports missing alt text on 53.1% of home pages; we saw it on 11%. Missing form labels: 51% there, 20% here. Buttons without an accessible name: 30.6% there, 12% to 13% here. Rule definitions differ, so read these as a direction, not exact gaps.

Security headers point the same way. About 39% of agency front pages sent a CSP and about 62% sent HSTS, both higher than the HTTP Archive Web Almanac 2025 reports for the web overall. Still, a majority had no CSP at all.

Tracking before consent

A third-party cookie was set before any consent choice on 21% to 22% of sites, and a tracking script loaded before consent on 21%. Our tallies don't record how many sites had both, so the safe statement is this: at least a fifth of agency front pages started tracking before the visitor chose anything, and the true share could be close to twice that.

Why deceptive design came back clean

Zero findings in 304 scans does not mean agencies never use dark patterns. Those patterns live in flows: checkouts, subscription forms, cancellation pages. An agency front page is mostly a portfolio, a services summary and a contact form, so there was very little for those rules to test. A score of 100 here means nothing was found where there was little to find.

Limits

This is one page, scanned once, by machine. No crawl, no logins, no clicks on consent banners, no form submissions. Sites that block automated visits were not scanned, so the sample favours agencies that are easy to find and allow scanning. A scan like this does not replace human review.

If you run an agency site

The three most common problems are also among the cheapest to fix: your grey text tokens, a Content-Security-Policy header and an HSTS header. The last two are server or CDN configuration, not page code.

Read the full paper

The full method, all 25 reported findings and the comparison with the earlier 57-site study are in the paper: Web Agency Front Pages at Scale: A Six-Dimension Benchmark of More Than 300 Sites.

DOI: 10.5281/zenodo.23986917. The earlier 57-site study: 10.5281/zenodo.23986811.

If you want to check your own front page, Sitejar has a free plan.