Best Hacker News Scrapers in 2026: Free API vs Paid Apify Actors

#webdev #javascript #programming #api

Finding the right Hacker News scraper can save you hours of manual data collection. Whether you're tracking startup trends, monitoring tech discussions, or building a news aggregator, here's a practical comparison of the best options in 2026.

1. Free Scraping API (frog03)

The simplest way to get started — no signup required:

curl "https://frog03-20494.wykr.es/api/v1/hn?q=startup&limit=10&api_key=demo-key-2026"

Returns clean JSON with title, URL, score, author, and timestamp. Supports query filtering and pagination.

Pros:

Free tier with demo key
No authentication setup
JSON response, ready to parse
Filters by keyword, date range, score

Cons:

Rate limited on demo key (100 req/day)
Best for prototyping and small projects

const response = await fetch(
  'https://frog03-20494.wykr.es/api/v1/hn?q=AI&limit=5&api_key=demo-key-2026'
);
const data = await response.json();
data.items.forEach(item => {
  console.log(`${item.title} (${item.score} points)`);
});

2. HN Algolia API (Direct)

Hacker News provides a search API through Algolia:

curl "https://hn.algolia.com/api/v1/search?query=startup&tags=story"

Pros:

Official, reliable
Full-text search
No API key needed

Cons:

Returns raw Algolia format (verbose)
No built-in filtering by score threshold
Rate limits apply
Requires parsing nested _highlightResult objects

3. Apify HN Scraper (Paid, Full-Scale)

For production workloads, the Hacker News Scraper on Apify handles everything:

Scrapes front page, newest, ask, show, jobs
Configurable depth (comments, user profiles)
Automatic retries and proxy rotation
Outputs to JSON, CSV, or direct dataset API
Scheduled runs via Apify platform

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_TOKEN' });
const run = await client.actor('cryptosignals/hackernews-scraper').call({
  section: 'front_page',
  maxItems: 100,
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();

Pricing: Pay-per-use on Apify. Free tier includes 5 USD/month of platform credits.

Comparison Table

Feature	Free API	HN Algolia	Apify Actor
Cost	Free (demo)	Free	Pay-per-use
Setup	None	None	Apify account
Search	Keyword filter	Full-text	Section-based
Comments	No	Yes (separate)	Yes (built-in)
Scheduling	No	No	Yes
Proxies	N/A	N/A	Included
Output	Clean JSON	Algolia format	JSON/CSV/API

Which Should You Use?

Start with the free API if you're prototyping or need quick keyword-filtered results. It returns clean JSON without any setup.

Use HN Algolia when you need full-text search across the entire HN archive, and you're comfortable parsing Algolia's response format.

Use the Apify actor when you need production reliability: scheduled scrapes, proxy rotation, comment extraction, and structured output at scale.

Quick Start: Build a Trend Tracker

Here's a minimal Node.js script that checks HN for trending AI topics every hour:

async function checkTrends() {
  const res = await fetch(
    'https://frog03-20494.wykr.es/api/v1/hn?q=AI&limit=20&api_key=demo-key-2026'
  );
  const data = await res.json();
  const hot = data.items.filter(i => i.score > 100);
  if (hot.length > 0) {
    console.log(`🔥 ${hot.length} trending AI posts on HN`);
    hot.forEach(i => console.log(`  ${i.score}pts: ${i.title}`));
  }
}
setInterval(checkTrends, 3600000);