Hacker News "Who is hiring?" to CSV: filter remote jobs by keyword and salary (Python, standard library only)

On the first weekday of every month, the whoishiring account opens an "Ask HN: Who is hiring?" thread, and a few hundred companies post one comment each. It's one of the best job boards for developers, but it's a single long page of free text. If you only care about remote roles using Rust or Python that pay at least $150k, you have to read all of it.

This post covers:

1. A standard-library Python script that fetches the latest thread from the Algolia HN API, filters remote posts by keyword and salary, and writes a CSV.
2. The pitfalls: free-text posts, funding rounds that look like salaries, "rust" matching "trust", and hourly versus yearly pay.

No account, API key or third-party package is needed.

1. A minimal version (Python standard library)

The Algolia HN Search API is free and needs no key. Two requests are enough: search_by_date with tags=story,author_whoishiring finds the newest thread, and /items/<id> returns the whole comment tree in one response. Job posts are the top-level comments. Their text is HTML (<p>, /), so the script strips tags and unescapes entities first.

"""Latest Hacker News "Who is hiring?" thread -> remote jobs with a salary -> hn_jobs_stdlib.csv.

Usage: python hn_hiring_stdlib.py python rust
"""
import csv
import html
import json
import re
import sys
import urllib.request

API = "https://hn.algolia.com/api/v1"
keywords = [k.lower() for k in sys.argv[1:]]


def get(url):
    req = urllib.request.Request(url, headers={"User-Agent": "hn-hiring-demo/1.0"})
    return json.load(urllib.request.urlopen(req, timeout=30))


# 1. newest "Ask HN: Who is hiring?" story posted by the whoishiring bot account
stories = get(f"{API}/search_by_date?tags=story,author_whoishiring&hitsPerPage=10")["hits"]
thread = next(s for s in stories if s["title"].startswith("Ask HN: Who is hiring?"))
print(thread["title"])

# 2. the whole comment tree in one request; job posts are the top-level comments
posts = [c for c in get(f"{API}/items/{thread['objectID']}")["children"] if c.get("text")]

SALARY = re.compile(r"[$€£]\s?\d[\d,.]*\s?[kK]?(?:\s?(?:-|–|to)\s?[$€£]?\s?\d[\d,.]*\s?[kK]?)?")
rows = []
for p in posts:
    text = html.unescape(re.sub(r"<p>", "\n", p["text"]))
    text = re.sub(r"<[^>]+>", "", text)
    first = text.split("\n", 1)[0]
    if not re.search(r"\bremote\b", text, re.I):
        continue
    if keywords and not any(re.search(rf"(?<!\w){re.escape(k)}(?!\w)", text, re.I) for k in keywords):
        continue  # whole words: "rust" must not match "trust"
    salaries = [m.group(0) for m in SALARY.finditer(text)
                if not re.match(r"\s?(?:m\b|mm\b|b\b|million|billion|in funding|series|raised|signing|bonus)",
                                text[m.end():m.end() + 15], re.I)]  # funding rounds are not salaries
    if not salaries:
        continue
    company = first.split("|")[0].strip() if "|" in first else "?"
    rows.append([company[:60], salaries[0].strip(), first[:200], f"https://news.ycombinator.com/item?id={p['id']}"])

with open("hn_jobs_stdlib.csv", "w", newline="", encoding="utf-8") as f:
    csv.writer(f).writerows([["company", "salary", "headline", "hnUrl"], *rows])
print(f"{len(posts)} job posts, {len(rows)} remote with a salary -> hn_jobs_stdlib.csv")
for r in rows[:5]:
    print(f"  {r[0][:28]:<28} {r[1]}")

Output (03.11.2026):

$ python hn_hiring_stdlib.py
Ask HN: Who is hiring? (November 2026)
157 job posts, 32 remote with a salary -> hn_jobs_stdlib.csv
  Simple Analytics             €5,000
  Bridge AI Consulting         $60K–$100K
  NEO Holdings (stealth, regul $200K to $300K
  ORFF Maestra                 $70–95
  FutureSearch                 $130k–$250k
$ python hn_hiring_stdlib.py python rust
Ask HN: Who is hiring? (November 2026)
157 job posts, 10 remote with a salary -> hn_jobs_stdlib.csv
  River                        $200K-$250K
  Paramark                     $199K–$249K
  Postscript (YC W19)          $180K to $220K
  SadServers                   $55 - $75
  Evaboot                      $60-120K

That runs in under a second. Look at the salaries, though: "€5,000" is a monthly salary and "$70–95" is an hourly rate. A plain "salary ≥ 150k" filter on these strings won't work.

2. Pitfalls

• It's free text. Most posts follow the Company | Role | Location | Remote | Salary | URL convention, but many don't ("Acme is hiring engineers in Berlin…"). Only take the company from the first segment when the headline uses pipes, and accept that some fields stay empty.
• Funding amounts look like salaries. "We raised $20M" and "$10k signing bonus" both match a naive currency regex. Check the words after the number (M, million, series, raised, bonus) and drop those matches.
• Pay comes in different units. "$55 - $75" is hourly, "€5,000" is monthly, "$150-200k" is a range with a single k. Before you compare anything, parse each salary into min/max, guess the period (very small numbers are hourly, four-digit amounts are usually monthly), and annualise it (×2080 for hourly, ×12 for monthly). Currencies are a separate problem. Converting them with today's exchange rate is fine for filtering, but keep the original string.
• Keywords need word boundaries. A substring search for "rust" matches "trust" and "go" matches "good". Match whole words ((?<!\w)rust(?!\w)), and expect short names like "Go" or "R" to stay noisy anyway.
• "Remote" can mean "not remote". "No remote", "remote: no", "remote (US only)" and "hybrid, remote possible" all contain the word. Read the headline first, look for negations, and keep the location next to the remote flag (e.g. Remote (US only)).
• Stay on the right thread. The same account also posts "Who wants to be hired?" (job seekers' personal details) and "Freelancer? Seeking freelancer?" threads. The script picks the "Who is hiring?" title explicitly. Don't collect the job seekers' thread into a database.
• API etiquette. The Algolia API is generous but not unlimited. One thread is one request. Cache it, because posts barely change after the first two days.

3. Running it every month

• Run the script on the 2nd or 3rd of the month (cron, a scheduled GitHub Action, or by hand); the thread is mostly complete by then.
• To cover several months, fetch the last few whoishiring stories instead of only the newest one and deduplicate by company, because many companies repost every month.
• Keep the raw comment text in the CSV next to the parsed fields, so you can fix parsing mistakes later without fetching again.

Wrap-up

• One thread, quick filter: two Algolia requests and the standard library. Use whole-word matching, drop funding amounts, and annualise salaries before comparing them.
• Every month: schedule the same script and deduplicate by company.

Questions and salary formats the script gets wrong are welcome in the comments.

Comments

[Lv Shao Mu]

This is a good write-up, and the funding-vs-salary filter is the detail most people would miss. I ran your regex over the November thread to see how the magnitude-based period guess holds up, and two cases are worth adding to your pitfalls list.

What I measured. 202 top-level posts, 111 salary-shaped matches. Classifying by the leading number (≤500 hourly, <20000 monthly, else annual) gives 13 hourly, 8 monthly, 90 annual. Most of that is right. The failures are consistent, not random:

1. A currency-prefixed value with a k suffix breaks the magnitude guess.

€45k-65k and £30k both parse to a small leading number (45 and 30) and land in the "hourly" bucket, when they are clearly annual. The problem is that your magnitude heuristic reads the digits, but the k is what carries the scale, and it sits on the second half of the range. Any rule of the form "small number → hourly" fails on £30k, €45k, $50k.

The fix is to normalise before classifying: if the string contains k or K, multiply the leading number by 1000 first, then apply the magnitude test. Only then does "≤500 means hourly" become safe, because a genuine hourly rate is never written $75k.

2. The $ prefix on the second half of a range is optional, and that's where the parse gets ambiguous.

$70–95, $75–100, $55 - $75, $23-34, $25-$30 — five of the eight "monthly" hits are this shape. Some of those are genuinely hourly consulting rates, some are monthly stipends. The number alone cannot tell you, and neither can the currency. What does tell you is the surrounding words: per hour, /hr, hourly, per month, /mo, monthly, annual, per year. Those appear often enough in this thread to be worth a second regex pass, and when none of them appear, the honest answer is to leave the period unknown rather than guess.

That matters for your stated goal, because a >= 150k filter silently drops €5,000/month (which annualises to 60k, correctly excluded) and silently keeps $55 - $75 if your guess said annual. Wrong in both directions, no error.

A third thing that isn't in the pitfalls list but bit me when I did the same thing:

€5,000/month and €5,000 are the same regex match but different jobs. Your example output shows Simple Analytics €5,000 — and the raw post says €5,000/month (full-time). The period is in the text, right after the number, and your current filter reads the 15 characters after the match to look for funding words, so it has the opportunity to see /month too. Adding the period words to that same lookahead is a one-line change and fixes the monthly case without a second pass.

On the annualisation constants: 2080 hours is right for a 40-hour week, but the same post above advertises "32h, 24h possible". If you annualise that hourly rate at 2080 you overstate it by 30% against someone working 32 hours. For filtering purposes it doesn't matter much; for a "is this better than my current job" comparison it does. Worth printing the assumed hours next to the annualised figure so the number is falsifiable.

One structural suggestion. You already keep the raw comment text in the CSV, which is the right call. I'd add one more column: period_source — one of explicit (text said /month), inferred (magnitude only), unknown. Then a reader can filter to explicit when they care about accuracy and accept inferred when they just want a rough list. It costs nothing and it turns the guessing into a documented field instead of hidden behaviour.

The rest — whole-word matching for rust/trust, the whoishiring thread selection, and caching the thread after two days — all matches what I found doing the same pulls. The Algolia response for November 2026 returns 202 top-level comments, all with text, so children filtering is safe here.