DEV Community: Justinnnnnnn045

My leaderboard site updates itself while I sleep — pure-stdlib Python, GitHub Actions, zero servers

Justinnnnnnn045 — Mon, 07 Sep 2026 10:12:43 +0000

My side project, PD.Radar, ranks resources mentioned in big Ask HN threads by citations (distinct commenters naming the thing), not by upvotes. Upvotes measure drama; citations measure how many people independently bothered to name the same resource.

The site went live with one leaderboard: 540 comments from "What is the best money you have spent on professional development?", 24 resources ranked. Therapy won with 51 citations. Not a course, not a book.

Then came the problem every static site has: the data was frozen on the day I built it. The fix wasn't updating it by hand every morning — that's a hobby for people with more patience than me. The fix was making the site mine its own new content.

The engine: two jobs, one file, zero dependencies

auto_update.py is ~300 lines of pure stdlib Python. It runs daily on GitHub Actions and does exactly two things:

Re-mine the flagship thread so its numbers stay fresh (and only rewrites the JSON if something actually changed)
Discover new Ask HN question threads, mine them with the same method, and publish the ones that qualify as new leaderboards

The site reads a single data/threads.json index. New file in, new tab on the site. No manual commits, no code changes, no server.

MIN_POINTS = 80
MIN_COMMENTS = 80
MIN_RESOURCES = 5      # a new thread needs >=5 resources matched
MIN_CITES = 3          # ...each with >=3 citations, to publish
MAX_NEW_PER_RUN = 3    # gentle on the Algolia API; quality over volume
SKIP_DAYS = 45         # days to remember a rejected thread before re-checking

The hard part was the honesty gate, not the mining

Mining is easy — HN has the excellent Algolia API, and fetching every comment of a thread is a loop. The hard part: how does a script decide a thread is worth publishing without me looking at it?

My first fear was false positives. A rant like "Is all of FAANG like this?" has hundreds of comments mentioning Python, Go, books, courses — the lexicon would happily produce a garbage leaderboard out of pure noise. Publishing that would poison the whole premise, because the site's entire promise is verifiable rankings.

So the gate is two lists:

INTENT_KEYWORDS = ["best", "book", "books", "recommend", "course", "learn",
                   "resource", "resources", "worth", "advice", "skill",
                   "skills", "career", "study"]
RANT_PATTERNS = ["like this", "rant", "vent", "just me", "wtf", "nuke",
                 "stopped", "banned", "shut down", "died", "layoff"]

def title_has_intent(title):
    t = title.lower()
    if any(p in t for p in RANT_PATTERNS):
        return False
    return any(k in t for k in INTENT_KEYWORDS)

A thread is only mined if its title asks for recommendations, and a discovered thread only gets published if the lexicon genuinely matches it: at least 5 distinct resources, each with at least 3 separate citations. Weak matches never see the light of day. And everything that gets rejected is remembered for 45 days, so the same thread doesn't get re-evaluated every morning.

It also runs the same citation matcher that powers the flagship (extract_rank.py), on purpose:

# Only import the lexicon + matchers from the existing pipeline so the
# ranking method stays IDENTICAL across threads.
from extract_rank import LEXICON, matches, flatten_comments, clean, load_flat

One method, every thread. If I ever improve the matcher, every leaderboard improves with it — and stays comparable.

What actually happened on the first runs

Discovery queries like "best books", "worth paying for", "best investment in your career" pull candidate threads from Algolia search. Each pass examined dozens of candidates. The first real run:

Thread	Points	Comments	Resources ranked
Best money spent on professional development (flagship)	524	540	24
Best book / resources on leadership for tech teams?	882	209	14
Best books you read in the past decade?	873	417	13
Advice for finding an entry-level remote job?	740	199	8
Best computer science book you've read recently?	670	186	7

1,551 comments, 66 ranked resources — none of it hand-mined. The flagship is pinned as the site's default tab; the rest sort by points.

And the rejections are as satisfying as the additions. The run that found the leadership thread also looked at a FAANG rant, a Stripe outage story, and a YC blog post — and correctly declined all three. Watching a script make a defensible editorial call with nobody watching is genuinely the most fun I've had with a side project this year.

Why GitHub Actions and not a server

Because the whole thing is a static site, the engine's output is just files. So the deploy pipeline is embarrassingly simple:

Cron fires at 06:17 UTC on a free GitHub Actions runner
The script fetches from Algolia, mines, decides
If anything changed: commit + push (as a bot user)
GitHub Pages redeploys automatically

Total monthly cost: $0. Total infrastructure: a YAML file that's 20 lines long. The commit history doubles as a changelog of every autonomous editorial decision the engine has made.

Honest limits

The lexicon is hand-built. It knows ~90 resources across books, courses, tools. A great comment recommending something it doesn't know is invisible to the ranking. Growing the lexicon is ongoing, boring, necessary work.
Discovery depends on Algolia search. If a qualifying thread never surfaces in my queries, it never gets mined. The 7 discovery queries cast a decent net, but it's a net, not a census.
Citations ≠ quality. The ranking proves what a community mentioned, not what changed anyone's life. The #1 professional-development answer being "therapy" (51 citations) is probably the most honest thing on the whole site precisely because no marketing budget could have produced it.

Try it

The site: PD.Radar — ranked answers from real threads — five leaderboards live, each with the receipts (every resource links back to actual comments in its source thread).

The engine: auto_update.py in the repo — steal the honesty-gate pattern if you're building anything that publishes without supervision. The gate is the product.

If you've got an Ask HN thread you'd love to see ranked this way, drop the link in the comments — the engine takes requests now.

The note-taking stack is broken: Anki still can't talk to your notes (and why that's a product gap)

Justinnnnnnn045 — Sat, 05 Sep 2026 10:04:04 +0000

This week a Hacker News thread made the rounds again: "Ask HN: What tools do you use to take learning notes and organise them?" (28 points, 26 comments).

The thread itself is not huge. But the pain in it is old, specific, and still unsolved — and that mismatch is exactly where a product gap hides.

The actual complaint

The OP is a self-directed learner. They use Anki for spaced repetition and something else (OneNote, Evernote, Obsidian, plain markdown) for long-form notes. And they are stuck in the glue between the two:

"Anki is useful but its interface is stuck in the 90s and it doesn't easily sync with my other, more expansive notes."

That's not a feature request. It's a description of a workflow that has one job — remember what you learn — and it fails at the seam between its two halves.

Why this is a real gap and not just nostalgia

The two tools got better separately, never together. Obsidian grew graph views and a plugin ecosystem. Anki grew better scheduling. Nobody made the bridge feel native.
The people who want this are skilled, motivated, and organised enough to pay. Self-directed learners are the exact demographic that buys Pro tools, subscriptions, and plugins for their workflow.
The existing bridges are brittle. Most are scripts, CSV exports, or add-ons that break on updates. The pain persists because the solution is treated as a weekend script, not a product.

What a product would actually look like

A markdown vault + spaced repetition in one place — not two windows and a sync script, but one surface where #flashcard blocks are just part of the note.
Sync that doesn't rot. The workflow survives app updates because the storage format is plain files.
Zero-config setup for the 90% case. Import a folder, get a deck. No CSV, no add-on fiddling.

Why this matters for builders

This thread is a small-but-perfect example of how to read a market: a repeated, specific complaint with a committed audience and a brittle existing answer. Not every opportunity is 500 points on HN. Some are 26 comments long and still the right thing to build.

This week's findings come from the daily opportunity brief generated from live Hacker News and Stack Exchange signals.

What's your note stack? Is Anki the missing link or the wrong tool? Tell me in the comments.

The #1 professional-development spend of 540 engineers isn't a book

Justinnnnnnn045 — Thu, 03 Sep 2026 23:03:34 +0000

I spent a weekend reading all 540 comments from Hacker News' biggest Ask HN on professional development: "What is the best money you have spent on professional development?" (524 points, 540 comments).

The results surprised me — and they might change where you put your next learning budget.

Update (Sep 7): this leaderboard no longer covers one thread. The engine now mines qualifying Ask HN threads on its own and publishes them daily — the site is up to 5 leaderboards, 66 resources, 1,551 comments, zero hand-mining. Details on how that works (and why it refuses to publish junk) in the follow-up article.

The ranking (by how many DISTINCT commenters named it)

#	Resource	Mentions	Est. price
1	Therapy / coaching	51	~$100/mo
2	Writing / blogging practice	13	Free
3	Refactoring (Martin Fowler)	4	~$45
4	Designing Data-Intensive Applications	3	~$45
5	Domain-Driven Design (Eric Evans)	3	~$55
6	Toastmasters	3	~$10
7	Coursera Specializations	2	~$50
8	LeetCode	2	~$35
9	MasterClass	2	~$15
10	The Pragmatic Programmer	2	~$45

The finding that surprised everyone

Therapy is #1. Not a book, not a course, not a certification — the highest-ROI professional-development spend according to 51 distinct engineers was coaching/therapy. The comments overwhelmingly said it unlocked career growth more than any technical book:

"As a founder / CTO the best money I have spent is on a really good coach." — gtf21

The technical books are still there (Refactoring, DDIA, DDD), but the pattern is clear: once the fundamentals are covered, the biggest gains come from non-technical skills — communication, leadership, and mental health.

The other four leaderboards back the same pattern from different angles — the leadership-resources thread (882 points) and the decade-of-books thread (873 points) both rank, among real practitioners, communication and judgment books above tools.

Methodology (what makes this honest)

A 524-point Ask HN thread asked the question and got 540 answers.
I read all 540 comments and ranked every resource by how many distinct people named it.
Not ranked by marketing. Not ranked by affiliate commission. Ranked by what engineers actually paid for and recommended.
Prices are labeled as estimates. No fake numbers, ever.

The interactive version

66 resources across 5 threads, grouped by price tier, searchable, with real quotes behind every item — every ranking verifiable against its source thread in seconds:

https://justinnnnnnn045.github.io/pd-engine/

What did YOU spend PD money on that's missing from the list?