DEV Community

Project Ace
Project Ace

Posted on

Monitoring Hacker News for your keywords? Most "hits" don't mention them

#hackernews #api #javascript #monitoring

If you track mentions of your product, framework or competitor on Hacker News, you probably use the free Algolia HN Search API. It's great: fast, free, no key. But it has a catch that is easy to miss.

The numbers

I searched HN comments since 1 October 2026 with search_by_date and then checked each result for the keyword itself:

Keyword Hits reported by Algolia Comments that actually contain the word
rust 11,133 1,715 (15 %)
deno 4,822 443 (9 %)
postgres 608 582 (96 %)

Short words are the problem. Part of it is that Algolia also searches author names (rust returns comments by a user called rustyhancock), and it is typo-tolerant. But for deno, almost none of the extra hits contain anything that looks like "deno" at all, and I could not pin down why. Distinctive words like postgres are mostly fine.

If you count mentions, build a dashboard or feed comments into an LLM for sentiment, 85–90 % noise ruins the result.

Why the obvious fix doesn't work

My first idea was to use Algolia's _highlightResult: keep a hit if any field has matchLevel != "none". That looked perfect in a quick test, but Algolia only returns highlight data for roughly the first 70 hits of a response. Everything after that has no highlights, so this filter silently throws away most of the good results too (for postgres it kept 74 of 717 real matches). I shipped that bug for about an hour before noticing.

What works

Check the text yourself. Keep a hit only if every keyword appears at the start of a word in the title, comment text or URL:

const words = query.toLowerCase().split(/\s+/).filter(Boolean);
const res = words.map((w) => new RegExp(`(^|[^a-z0-9])${w.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')}`, 'i'));

function isRealMatch(hit) {
  const text = `${hit.title ?? ''} ${hit.story_title ?? ''} ${hit.comment_text ?? ''} ${hit.url ?? ''}`
    .replace(/<[^>]+>/g, ' ');
  return res.every((re) => re.test(text));
}
Enter fullscreen mode Exit fullscreen mode

"Start of a word" keeps postgres → PostgreSQL but drops rust → trust.

Two more things you'll hit when collecting more than a few hundred items:

  • The 1,000-hit cap. Algolia returns at most 1,000 results per query. With search_by_date, take the oldest created_at_i of each page and query again with numericFilters=created_at_i<{oldest} to walk back in time.
  • Filtering shrinks pages. After filtering, a page of 1,000 hits might give you 80 items, so keep walking until you have enough.

Disclosure: This article was written with help from an AI assistant (Claude); the numbers come from my own queries on 18 October 2026.

Top comments (0)