DEV Community

Addtocalendarlab
Addtocalendarlab

Posted on

We built a comparison site where you can recompute every score yourself

Most comparison sites rank products partly by commission, and from the outside you can't tell how much. We wanted to build one where you can check the scores instead of trusting us.

The niche is small: add-to-calendar tools, the services behind "Add to calendar" buttons on event pages and in emails. We rank six and list a few more.

Stack

Astro with static output, Tailwind v4, and one Cloudflare Worker that serves the files and a couple of API routes. Tool data is one YAML file per tool in an Astro content collection. Reviews and guides are MDX. Scoring is tested with Vitest.

Sources

Each tool's YAML holds its plans, prices, limits and features. The content schema (Zod) rejects a claim without a source URL. A script saves every source page to the Internet Archive and writes the snapshot link back into the YAML, so when a vendor edits their page we still have what it said when we read it.

Features also have an evidence level. Verified means we tested it, and the schema requires a date. Documented means the vendor says so. Unclear means the vendor's docs are vague, and needs a note explaining why. Nearly everything is documented, because hands-on testing is slow, and we'd rather show that than label things verified that weren't.

Scoring

All point values are in rubric.ts. The scoring functions in scoring.ts are pure, and no other file is allowed a number that affects a score. The methodology page reads the same file.

At build time those two files go through TypeScript's transpileModule and are published as /data/scoring.js, next to /data/tools.json. A short example script downloads both and recalculates every score.

After the build, a check runs the published scoring.js against the published tools.json. If any result differs from the score on the pages, the deploy fails. Hand-editing a score would break it.

Other weightings

Someone will always disagree with the weights. The methodology page reranks every tool under 15 other weightings, such as double weight on price or none on support, and shows each tool's best and worst place.

There's also /api/weights?price=2&support=0, which returns the ranking with your multipliers as Markdown or JSON. We added it after ChatGPT failed to rerank the tools from the methodology page. It could read the page but couldn't run the JavaScript tool on it. A URL it can fetch works. The handler is a few dozen lines in the Worker and uses the same scoring code.

Prices

A second Worker runs every Monday. Two vendors embed their prices as data in the page, so we read those directly. For the others it loads the pricing page in Cloudflare Browser Rendering and Workers AI extracts the prices. The currency is taken from the symbols on the page, not from the model.

Each check is a row in D1 with the page text kept as evidence. A changed price doesn't go live by itself. It opens an issue, someone confirms it on the vendor's site, updates the YAML and adds an entry to the public changelog. When nothing changed, the "prices checked" date moves forward and the site rebuilds.

We also rebuilt older prices from Wayback Machine snapshots, back to 2018 for one vendor. Pricing pages with a monthly/yearly toggle were often archived showing yearly prices only. For one tool the monthly prices survived in an archived JavaScript bundle (value ? "9.60" : "12"). Where the bundle wasn't archived, the table says the price is unknown.

Corrections

Every price, score or rubric change is a YAML file in corrections/, rendered as a public changelog. Readers and vendors can report mistakes through a form, which the Worker turns into an issue in a private repo.

Markdown copies

The build also writes a Markdown copy of every page, served when a request sends Accept: text/markdown. There's an llms.txt, an OpenAPI file for the weights endpoint, and CORS on the data files. Every page was already generated from data, so these were small additions.

Affiliate links

Some tools pay us a commission, and every page says so. The post-build check fails if an affiliate link lacks rel="sponsored" or its label. Because the scoring code is public, a commission can't move a tool up without it showing.

What we'd keep

Make the build fail on mistakes you'd be embarrassed to publish: scores that don't match, missing sources, unlabelled links. Keep every number in one file, so the page, the API and the published script can't drift apart. And write "unknown" when you don't know.

The site is Add to Calendar Lab. Questions welcome in the comments.

Top comments (1)

Collapse
 
a2clab profile image
Addtocalendarlab •

Any questions? Send below and I will reply to you 🔥