Extracting design tokens from a site you don't control

The problem: given a URL, produce a design token set — colors, type scale, spacing rhythm, border radii, logo — good enough that a generic app re-skinned with those tokens reads as belonging to that brand.

This sounds like a scraping problem. It is mostly not. Scraping gets you a pile of CSS values in about an hour. Turning that pile into a usable token set is where the actual work is, and it took me several iterations to get right. Here is the approach that ended up working.

Why the naive version fails

The obvious first attempt: load the page, walk the DOM, collect every computed style, count frequencies, take the top values.

I built this. It produces garbage, for a reason that is obvious in retrospect. Frequency counting is dominated by whatever element type is most numerous on the page, which is almost always body text and its containers. You get seven shades of near-black, #ffffff, one grey, and a 16px font size. Technically accurate. Completely useless for making something look like the brand, because a brand's identity lives in its accents — the button color, the display heading, the one saturated hue used sparingly — and accents are by definition rare.

The fix is to weight by visual prominence rather than count.

Weighting by prominence

Every element gets a prominence score before its styles are counted. The score I converged on:

prominence = area_ratio × viewport_weight × role_weight

area_ratio is the element's rendered area over the viewport area. A hero section outweighs a footer link.

viewport_weight decays with distance below the fold. Something at scroll position zero counts fully; something 4000px down counts for very little. Brands front-load identity.

role_weight is a lookup by tag and semantic role. Buttons, links, headings, and elements with role="button" get multiplied up substantially. This single term did more for output quality than anything else I tried, because it surfaces exactly the accent colors that frequency counting buries.

With prominence weighting, a color used on three buttons above the fold outranks a grey used on four hundred spans, which is the correct answer.

Color: cluster, then assign roles

Raw color extraction returns hundreds of near-identical values — anti-aliasing, opacity compositing, and hover states generate a smear around every intended color.

Convert to a perceptually uniform space first. Clustering in sRGB gives you results that look wrong to a human eye because sRGB distance doesn't match perceived difference. OKLCH fixed this for me; LAB works too. Cluster with a distance threshold rather than a fixed k — you don't know in advance whether a brand has three colors or eleven.

Then the harder part: assigning roles. A cluster set is not a token set. You need to know which one is primary, which is background, which is text.

Heuristics that held up:

• Background is the highest-area cluster, near-always. Very high confidence.
• Text is the cluster with maximum contrast ratio against background, weighted toward clusters that appear on <p> and <li> elements.
• Primary is the highest-chroma cluster that appears on an interactive element. Chroma is doing the heavy lifting — brands reserve saturation for the thing they want clicked.
• Surface is a cluster within a small perceptual distance of background but not identical. Cards, panels, elevated regions.

Anything left over goes into an unassigned pool. I do not try to force-assign; a wrong role assignment is worse than a missing one, because downstream code can fall back gracefully on a missing token but will render confidently wrong output on a mislabeled one.

Type scale: fit a ratio, don't collect sizes

Collecting font sizes gives you a list like [14, 16, 16, 18, 24, 32, 48]. You can use it directly, but the output is brittle — any size you need that isn't in the list has no principled value.

Better: fit a modular scale. Take the prominence-weighted mode as the base (almost always the body size), then regress the remaining sizes against candidate ratios — 1.125, 1.2, 1.25, 1.333, 1.5, 1.618 — and pick the best fit by residual.

Now instead of seven fixed numbers you have base and ratio, and every step is derivable. When the demo needs a size the source site never used, you generate one that is consistent with the brand's rhythm rather than guessing.

The same treatment works for spacing, though the fit is noisier because spacing is more often ad hoc. I extract a base unit — usually 4px or 8px, found as the GCD of common margin and padding values after discarding outliers — and express everything as multiples.

Border radius is a personality variable

Underrated signal. Collect radii from cards, buttons, and inputs, weighted by prominence, and reduce to a small set.

What matters is not the exact pixel value but where the brand sits on the sharp-to-round axis, because that position implies other decisions. Fully-rounded pill buttons want more horizontal padding than square ones to look balanced. Sharp-cornered cards tolerate tighter gaps than heavily-rounded ones. I derive a roundness scalar from the extracted radii and feed it into padding and gap calculations rather than treating radius as an isolated token.

This is the difference between a re-skin that looks like the brand and one that looks like the brand's colors applied to someone else's layout.

Logos: prefer declared, verify rendered

Order of attempts:

1. <link rel="icon"> and its variants, preferring SVG, then largest PNG.
2. Open Graph and Twitter card images — often a logo lockup, sometimes a product shot, so this needs verification.
3. An <img> or inline <svg> inside <header> or <nav>, positioned in the first third horizontally. This is where a wordmark lives on the overwhelming majority of sites.
4. Anything with logo in a class, id, alt, or filename.

Each candidate gets checked: reasonable aspect ratio (reject anything near-square and tiny, which is usually an icon rather than a wordmark), reasonable dimensions, and not a photograph. A quick edge-density check separates vector-ish marks from photos well enough.

I extract the aspect ratio explicitly and store it as a token, because header height depends on it and a 6:1 wordmark and a 1:1 mark produce meaningfully different headers.

Screenshots are for verification, never extraction

The most useful architectural decision I made.

It is tempting to pull colors out of a screenshot — it works, it's simple, and it handles canvas and image-heavy sites where the DOM tells you nothing. But screenshot pixels have been through compositing, anti-aliasing, and JPEG artifacts. The color you sample is near the brand color, never equal to it. Downstream, near-correct tokens produce output that looks subtly wrong in a way that is very hard to debug.

The DOM has ground truth. Use it.

Screenshots earn their place at the end: render the themed output, screenshot it, diff it against the source screenshot, and use the mismatch as a quality signal. Sub-5% pixel mismatch means the token extraction did its job. A large mismatch localized to one region tells you which token is wrong — a header-region mismatch points at logo sizing, a body-wide one points at type scale.

Verification, not source. That distinction is the whole thing.

Failure modes worth knowing about

Heavy canvas or WebGL sites. The DOM is nearly empty. There is no good answer; these get flagged for manual handling rather than producing confident garbage.

Aggressive bot protection. Some fraction of sites will not load headless. Budget for a fallback fetching path and accept that a percentage will need a human.

Sites mid-rebrand. New homepage, old interior pages. Extracting from more than one URL and detecting disagreement catches this, but resolving it requires knowing which one is current, which requires a human.

Dark mode defaults. If the extraction runs with a dark color-scheme preference and the brand's primary identity is light, every token is inverted. Pin the preference explicitly.

The general lesson

The scraping is the easy part and it is not where the quality comes from. The quality comes from the interpretation layer — prominence weighting, role assignment, ratio fitting, deriving tokens from other tokens rather than collecting them independently.

Which is a fairly ordinary engineering lesson dressed in an unusual problem: the value is in the model you impose on the data, not in the collection of it.