DEV Community

Cover image for We scored 30 real websites against a 40-check design contract. Here's what we found.
LE-VAI
LE-VAI

Posted on • Originally published at github.com

We scored 30 real websites against a 40-check design contract. Here's what we found.

The headline

We scored 30 sites from our live leaderboard — design systems (Primer, Spectrum, Geist), SaaS (Vercel, Linear, Stripe), hardware (Apple), inspiration galleries (CSS Design Awards, FWA), and more.

Designesy runs 40 deterministic checks against any URL — tokens, motion, accessibility, cadence, takt, poise, identity, interaction, performance, responsive, semantic, security, spec, and copywriting. No LLM, no heuristics. Every check is reproducible.

The results were worse than we expected.

Metric Value
Sites scored 30
Score range 37.9 – 100.0
Mean score 62.2
Median score 62.4
Grade A 1 site (3%)
Grade B 0 sites (0%)
Grade C 6 sites (20%)
Grade D 11 sites (37%)
Grade F 12 sites (40%)

60% of sites scored D or F. Zero sites scored a B. The gap between "looks good" and "passes a deterministic design contract" is enormous.

What "pass" looks like

Across 30 sites × 40 checks = 1,110 total checks:

Result Count %
Pass 411 37.0%
Warn 506 45.6%
Fail 118 10.6%
Skip 75 6.8%

More checks warn than pass. A "warn" means the check detected something partial — a token file exists but uses bare hex instead of structured color, or reduced-motion is present but not tiered. The industry is in a half-adopted state: teams know they should have design tokens, motion standards, and accessibility gates, but most haven't finished the job.

Where sites fail the most

Dimension Avg score Sites below 50
Tokens 40.8 11/30
Motion 52.5 11/30
Takt 50.8 3/30
Accessibility 58.4 4/30
Interaction 60.0 12/30
Cadence 60.1 3/30
Poise 75.0 0/30
Identity 83.3 0/30

Tokens (avg 40.8 — the worst dimension)

11 of 30 sites score below 50 on token compliance. The common failures:

  • Bare hex colors instead of structured tokens ($type: color with colorSpace + components)
  • No $schema pointer — token files don't declare which spec they conform to
  • Magic numbers in spacing/radius instead of a named scale
  • No dark-mode surface tokens — sites that have dark mode but don't structure their surfaces as lightness-stepped tokens

This is the W3C DTCG 2025.10 gap. The spec went stable in October 2025, and 84% of teams say they use design tokens — but most are using them informally, not in a conformant format.

Motion (avg 52.5 — second worst)

11 of 30 sites score below 50 on motion. The common failures:

  • No reduced-motion pathprefers-reduced-motion is either absent or a kill switch that yanks all animation instead of tiering (Tier 1 remove / Tier 2 soften / Tier 3 keep)
  • No easing tokens — sites use 15 different cubic-beziers instead of a 4-token easing scale
  • Layout animation — animating width, height, top, left instead of transform/opacity
  • No duration scale — every transition is a different magic-number duration

Apple's HIG is the reference here — they tier reduced-motion (completely remove, soften, or keep) instead of treating accessibility as a binary kill switch. Only 1 site in our sample does this correctly.

Interaction (avg 60.0 — 12 sites below 50)

12 of 30 sites score below 50 on interaction. This is the dimension with the most outright failures:

  • :focus strips outline without replacement — the most common accessibility failure pattern
  • No :focus-visible — keyboard and mouse focus are treated identically
  • No press-settle — no scale(0.97) or equivalent press feedback on interactive elements

The top 5

Rank Score Grade Site Notable
1 100.0 A Designesy Self-scored — transparency earns trust
2 77.7 C Apple Tiered reduced-motion reference
3 77.6 C GitHub Primer 518 tokens — most in the sample
4 73.8 C zeroheight Design-system documentation tool
5 73.6 C Vercel Clean token + motion baseline

No site scored a B. The jump from C (top non-self score: 77.7) to A (100) is 22 points — that's the gap between "has tokens but uses bare hex" and "has structured DTCG-conformant tokens with a $schema pointer and colorSpace."

The bottom 5

Rank Score Grade Site Why
26 49.4 F Adobe Spectrum Tokens exist but not DTCG-conformant; motion not tiered
27 50.3 F Vercel Geist Surprising — Vercel's own design system scores lower than their marketing site
28 41.1 F Pentagram Agency site — beautiful but zero token/motion structure
29 41.1 F FWA Inspiration gallery — awards "good design" but fails the contract
30 37.9 F CSS Design Awards Same irony as FWA

The bottom 5 tells a story: the sites that award "good design" fail a deterministic design contract. CSS Design Awards and FWA exist to celebrate design excellence, but they score 37.9 and 41.1 — because their own sites don't use structured tokens, tiered reduced-motion, or DTCG-conformant formatting.

The Geist vs. Vercel split

The most interesting finding: Vercel's marketing site (vercel.com) scores 73.6 (C) while Vercel's design system (geist.dev) scores 50.3 (F). The marketing site outperforms the design system by 23 points. This means Vercel's marketing team is doing more design-system work than Vercel's design-system team — or that Geist's token format predates DTCG 2025.10 and hasn't been migrated.

This pattern repeats: marketing sites score higher than the design systems they're built from. Marketing teams optimize for visual polish. Design system teams optimize for API stability. The contract rewards the former.

What this means

  1. Design tokens are half-adopted. Everyone has them; few have them in a conformant format. The W3C DTCG 2025.10 spec is 10 months old and the ecosystem hasn't caught up.

  2. Motion accessibility is the biggest gap. 11/30 sites score below 50 on motion. The prefers-reduced-motion kill-switch pattern is everywhere — but tiered reduced-motion (the Apple HIG approach) is almost nowhere.

  3. "Good design" is not "systematic design." The sites that win design awards fail design contracts. Visual impression ≠ systemic rigor. A site can look stunning and still use bare hex, magic-number spacing, and no reduced-motion tier.

  4. No one has a B. The gap between C and A is 22 points. There's no middle ground — you're either doing tokens informally (C/D) or you're doing them conformantly (A). The B tier is empty because the transition from "informal tokens" to "DTCG-conformant tokens" is a cliff, not a slope.

Try it yourself

# Score any URL in 5 seconds:
npx designesy-score@latest https://your-site.com

# Or use the MCP server:
claude mcp add designesy --transport http https://www.designesy.org/api/mcp

# See the full leaderboard:
open https://www.designesy.org/leaderboard
Enter fullscreen mode Exit fullscreen mode

The scoring engine is deterministic, open, and free. No API key, no auth. The methodology is published, the contract is public, and the GitHub Action gates your CI on the same 40 checks.


Designesy is a design-system contract verification engine. 40 deterministic checks, A–F grade, DTCG token validation, WCAG 2.2 AA accessibility, Lottie motion validation. designesy.org.

Top comments (1)

Collapse
 
mads_hansen_27b33ebfee4c9 profile image
Mads Hansen

Reproducibility is a real strength here, but deterministic still means “conforms to this contract,” not automatically “good design.” The self-authored site scoring 100 and the empty B band look like useful calibration signals rather than conclusions by themselves.

I’d publish the contract/version and per-check evidence with every score, then run sensitivity analysis on weights and thresholds plus a blind comparison against independent accessibility/design reviewers. If modest weight changes reshuffle the leaderboard, the dimension profile is more informative than the letter grade. For CI, pin the contract version and make upgrades produce an explicit score diff, so a methodology release cannot become a surprise regression.