Three public HTTP APIs I read daily without registering for a key

My daily trends fetch is a Node.js script that runs in GitHub Actions, hits three APIs, and writes a JSON file that feeds my X-drafts pipeline later in the day. None of the three is supposed to require an API key. No OAuth flow, no dashboard signup, no rate-limit token to rotate. Two of them return data every morning; the third, Reddit's .json endpoints, has answered 403 since the collector's first run, which is its own kind of lesson.

That sounds trivial, but it matters for CI pipelines. Every API key stored in GitHub Secrets is a secret that can expire, a secret that has to be rotated when a team member leaves, a secret that creates a failure surface. Keyless reads eliminate all of that for the sources where it's possible.

Here are the three I wired in and the practical limits of each — the two that hold up daily, and the one that doesn't.

Hacker News Firebase API

The HN API is publicly documented on GitHub and hosted on Firebase. No key, no auth header, no rate limit published in the official docs.

const ids = await fetchJSON(
  "https://hacker-news.firebaseio.com/v0/topstories.json"
);
const items = await Promise.all(
  ids.slice(0, 20).map((id) =>
    fetchJSON(`https://hacker-news.firebaseio.com/v0/item/${id}.json`)
  )
);

This returns the top story IDs, then fetches each item. The item objects include title, url, score, and descendants (comment count). For my use case — grabbing the top 20 stories to identify what's trending in dev — that's everything I need.

The practical limit is latency. Fetching 20 items individually over Firebase takes 2–4 seconds depending on cold-start behavior. Parallelizing with Promise.all handles this fine in a script context. In a browser context, sequential fetches would be painful.

Firebase is reported to throttle aggressively for a few minutes if you hit it from many IPs in a short window. Since I started this collector in May 2026 I haven't seen it happen: the script still fires all 20 item requests concurrently through Promise.all with no delay between them. The only spacing is at the workflow level — a random 0–3 minute sleep before the run so the job isn't starting on the same second as every other cron on the hour.

dev.to public API

dev.to exposes a read-only articles endpoint that works without authentication for public content. The top=1 parameter returns articles sorted by recent reaction count.

const items = await fetchJSON(
  "https://dev.to/api/articles?per_page=12&top=1"
).catch(() => []);

The response includes title, url, public_reactions_count, comments_count, and tag_list on each article. The catch(() => []) is load-bearing: dev.to's API returns 503 occasionally, and swallowing that to an empty array lets the rest of the trends fetch continue.

One thing that surprised me: top=1 doesn't mean "top from the last 1 day". The documentation is ambiguous about the time window. In practice, articles from the previous two to three days appear in the results. For trending detection that's fine; for building a "published today" feed it would be wrong.

The API does not require auth for reading public articles. It has a rate limit — 10 requests per second per IP, per the docs — which I've never come close to hitting with a once-daily run.

Reddit .json endpoints

Reddit's most underused feature is the .json suffix that works on almost any listing URL: append .json to a subreddit URL and you're supposed to get the full listing data without OAuth. This is the source that didn't work out for me, so here's the setup first and the reality after it.

const j = await fetchJSON(
  "https://www.reddit.com/r/programming/top.json?t=day&limit=10"
).catch(() => null);

The response structure is {data: {children: [{data: {title, url, score, ...}}]}}. Each child has title, url, score, num_comments, author, and selftext (body for self-posts).

The User-Agent header matters here. Reddit blocks requests without a user agent string, and returns 429 for user agents that look like bots making too many rapid requests. I send hogwartz-trends-fetch/1.0 as the UA string.

const r = await fetch(url, { headers: { "user-agent": "hogwartz-trends-fetch/1.0" } });

That's the theory of the .json suffix, and it's why I wired Reddit in. It is not what my pipeline actually got. Every snapshot the collector has written since its first run on 2026-05-10 records zero Reddit items: https://www.reddit.com/r/<sub>/top.json answers 403 to unauthenticated clients, and the .catch(() => null) above turns that into an empty list, so the only trace in the output file is "r/programming": 0 in the counts block — no status code, no error, nothing that reads as a failure. So of the three sources here, this is the one that did not hold up for me: HN and dev.to have returned data every day, Reddit has returned none.

(Update, 2026-08-12: I re-pointed the Reddit source at the old.reddit.com listing and made the fetcher record every source failure in an errors[] array in the output instead of swallowing it. old.reddit.com returns 403 to this collector too — the difference is that the snapshots now say so out loud.)

What I do with the combined output

The script writes a single JSON file to content/trends/YYYY-MM-DD.json with the sources merged and deduped by URL:

const all = [
  ...hn.map((x) => ({ ...x, source: "HN" })),
  ...devto.map((x) => ({ ...x, source: "dev.to" })),
  ...reddit.map((x) => ({ ...x, source: `r/${sub}` })),
];
await writeFile(OUT, JSON.stringify({ date: today, items: all }, null, 2));

The fetch workflow itself does nothing else: checkout, node, commit the file. A separate PMO Morning Brief routine picks the file up later the same morning and uses it to draft X posts via Claude, on its own Anthropic quota. The trends file is the handoff artifact between the two, which is why this repo's workflow needs no Anthropic key at all.

That separation also means I can test the collection step in isolation without spending any API budget. The trends fetch swallows individual source errors and still writes a file with whatever it managed to collect. Downstream steps get degraded input rather than a broken run — and the Reddit 403 above is exactly what that costs you: a zero in a counts field is the only signal that the input was degraded, and a zero is easy to read as "quiet day".

Where the keyless approach breaks down

Not every useful data source has a public read endpoint. GitHub's search API requires auth past 60 requests per hour. HuggingFace's models API requires a token for private repos but is keyless for public models — I use the keyless endpoint for that too, but it's limited to what HF considers "public". Steam's storefront API has undocumented endpoints that work without auth, though the behaviors are inconsistent enough that I treat them carefully.

Two of the three sources above — HN and dev.to — have been stable since I started pulling them in May 2026. Reddit .json, as above, was not. That's not a guarantee of permanence either. Reddit has changed its API policies before; dev.to's read rate limits could tighten. The fallback behavior (catch to empty array) means the pipeline degrades gracefully rather than breaking when that happens.

Part of an ongoing 6-month experiment running three AI-curated directory sites. The technical claims here are real; this article was AI-assisted.