On 1 Nov, Andrej Karpathy shared HackerTube (https://hackertube.org/). He had Claude filter 43M Hacker News comments from 20 years down to 6,454 recommended videos, and he said the site took about 2 hours with Opus 5.5 in Claude Code. He also published the data as a public zip.
As a Lead SWE, my first question was whether I could pull off the same trick with a different topic and a different tool. I picked books, because HN comment threads are basically one long reading list. I picked Cursor, because that's the tool I use every day.
The result is HackerShelf: https://ishankdev.github.io/hackershelf/ (code: https://github.com/IshankDev/hackershelf).
How Karpathy's pipeline works
His DATA.md lays out the method, and I copied it:
- Scan every HN comment from 2007 to 2026 in the open-index/hacker-news dataset for YouTube links. The comment needs at least 100 characters of its own text.
- Keep a video only if at least 3 different people mentioned it in at least 3 different months.
- Claude Sonnet 5.5 labels each mention with a stance, topics, a reason and a quote.
- Sonnet writes a card for each video, then puts it into 1 to 3 categories.
- Opus 5.5 in Claude Code runs the whole thing.
He did a few steps by hand: a Google Cloud API key for YouTube metadata, a VPS and a domain.
My version: HackerShelf
Same idea, but for books:
- Source: HN comments from Jan 2024 to Aug 2026 from open-index/hacker-news, 9,437,609 comments in total
- Pre-filter: DuckDB regex for book links (Amazon, Goodreads, Open Library, Bookshop, ISBNs, O'Reilly, Manning, PragProg, No Starch) and phrases like "must read", "worth a read" and "book called". This took about 9 seconds and left 11,519 candidates.
- Extraction: gemini-3.5-flash-lite in 288 batches of 40 found 6,076 book mentions.
- Resolve: Open Library matched 2,212 works. Using Karpathy's rule (at least 3 people, at least 3 months), 151 books were left.
- Cards: one Gemini call per book, so 151 calls
- Site: a static site on GitHub Pages
The final shelf has 151 books, 723 mentions and 601 unique recommenders, sorted into 9 categories.
Time: Karpathy's 2 hours vs. mine
Being honest, my total was longer than his.
- Planning and data checks: 7:47 to 7:50 PM IST
- Cursor build (pipeline code, the full site and deploy setup): about 40 minutes, 7:56 to 8:37 PM IST
- Gemini extraction: about 20 minutes, about 8:50 to 9:11 PM IST
- Open Library resolve and filter: about 44 minutes, 9:24 to 10:08 PM IST, with no LLM. This was the slowest step.
- Cards: 509 seconds, 10:09 to 10:17 PM IST
- Site data: under 1 second, at 10:17 PM IST
From start to finish that's about 2.5 hours of wall clock (7:47 to about 10:17 PM IST). That includes my own manual steps: creating the repo, giving Cursor GitHub access and turning on GitHub Pages.
Cursor's part, writing the actual code, took about 40 minutes. Most of the remaining time went to calling APIs and waiting on rate limits.
Cost
- Tokens: 1,968,177 in and 623,943 out
- What I paid: $0, on Gemini's free tier. I hit 429s and handled them with backoff and retry.
- Paid-rate equivalent: at $0.30 in and $2.50 out per 1M tokens, about $0.59 + $1.56 = about $2.15
- Cursor usage cost: I don't know this number yet, so I won't guess.
What went wrong
- The first Cursor run shipped sample data. The site worked, but it only had 9 placeholder books. The real data had to come from a separate run of the LLM stage.
- Too many candidates. The first broad filter matched about 392k comments that might mention a book, which was far too many for a free tier. Tighter link and phrase rules cut that down to 11,519.
- The model paraphrased. 470 of the extracted "quotes" weren't actually word for word in the source comment. I didn't trust them, so I blanked them.
- Open Library isn't perfect. One top-10 entry resolved as "1984 (adaptation)" by Michael Dean instead of Orwell's novel. Fuzzy title matching is the weak link here.
Does it hold up?
I spot-checked 10 books and 83 mentions against the original HN comments. 83 out of 83 had the right author, and every quote was verbatim in its source comment. For example:
- "I recommend reading The Mom Test, it's a great book about how to navigate this early validation stage"
- "Robert Nystrom's Crafting Interpreters [0] is a really fantastic book on the topic."
The top 10 books Hacker News recommends (2024 to 2026)
- Designing Data-Intensive Applications, Martin Kleppmann: 13 recommenders, 14 mentions
- Amusing Ourselves to Death, Neil Postman: 11 recommenders, 11 mentions
- The Making of the Atomic Bomb, Richard Rhodes: 11 recommenders, 12 mentions
- Permutation City, Greg Egan: 11 recommenders, 11 mentions
- The Mom Test, Rob Fitzpatrick: 10 recommenders, 10 mentions
- A Philosophy of Software Design, John Ousterhout: 9 recommenders, 11 mentions
- Blindsight, Peter Watts: 11 recommenders, 11 mentions
- Chip War, Chris Miller: 9 recommenders, 9 mentions
- 1984: 10 recommenders, 10 mentions (Open Library mismatched this one, see above)
- Children of Time, Adrian Tchaikovsky: 11 recommenders, 11 mentions
The ranking uses a score that weighs strong recommendations above passing mentions, which is why a book with 9 recommenders can rank above one with 11.
What I took away
- Karpathy's recipe works beyond videos. The "3 people, 3 months" rule is a surprisingly good filter for noise.
- An AI coding tool can write the code in under an hour. The time goes into data, rate limits and the manual steps.
- Don't trust LLM-extracted quotes until you check them against the source. 470 were quietly wrong.
Credits
All credit for the idea and method goes to Andrej Karpathy and HackerTube. The comment data comes from open-index/hacker-news under ODC-By 1.0. The comments themselves are still their authors' work, and every mention on HackerShelf links back to the original comment.
Originally published on Medium.
Top comments (0)