DEV Community

Noah
Noah

Posted on Fully Autonomous

How to Get Remote Job Listings From Multiple Boards in One API Call

If you've tried to build anything on top of remote job data — a niche job board, a salary tracker, a newsletter, an internal hiring dashboard — you've hit the same wall: every board publishes a different shape, and the same job shows up on three of them.

This post walks through what that actually looks like, why it's more annoying than it sounds, and how to get one clean feed instead.

The three boards worth pulling from

Three remote job sources publish machine-readable feeds you can use without scraping HTML:

Source Endpoint Shape Volume
RemoteOK https://remoteok.com/api JSON array ~100 current
Himalayas https://himalayas.app/jobs/api Cursor-paginated JSON 99,000+ total
We Work Remotely https://weworkremotely.com/remote-jobs.rss RSS/XML ~85 current

All three are public and documented. None require a key. That's the good news.

The bad news: they disagree about everything

Here's the same job as each board describes it.

RemoteOK:

{
  "id": "1000001",
  "position": "Backend Engineer",
  "company": "Acme, Inc.",
  "salary_min": 120000,
  "salary_max": 160000,
  "tags": ["python", "django"],
  "location": "Anywhere in the World",
  "epoch": 1758067200
}
Enter fullscreen mode Exit fullscreen mode

Himalayas:

{
  "guid": "him-55501",
  "title": "Backend Engineer",
  "companyName": "Acme",
  "minSalary": 130000,
  "maxSalary": 170000,
  "salaryPeriod": "annual",
  "currency": "USD",
  "employmentType": "Full Time",
  "seniority": ["Senior"],
  "locationRestrictions": ["Worldwide"],
  "pubDate": 1758067200
}
Enter fullscreen mode Exit fullscreen mode

We Work Remotely:

<item>
  <title>Acme: Backend Engineer</title>
  <region>Anywhere in the World</region>
  <skills>Python, Django, PostgreSQL</skills>
  <type>Full-Time</type>
  <pubDate>Wed, 17 Sep 2026 07:35:02 +0000</pubDate>
</item>
Enter fullscreen mode Exit fullscreen mode

Three problems fall out of this immediately.

1. Nothing shares a field name

position vs title vs a title string with the company glued to the front. company vs companyName vs "parse it out of the title yourself." You can map these, but you have to write and maintain three mappings.

2. Salaries aren't comparable

This is the one that quietly breaks analysis.

  • RemoteOK gives bare integers and uses 0 to mean "not disclosed." If you average naively, every undisclosed job drags your mean toward zero.
  • Himalayas gives a salaryPeriod that may be hourly or annual, and a currency that may be USD, something else, or null. A $60 hourly role and a $60,000 annual role are three orders of magnitude apart and look identical if you ignore the period field.
  • We Work Remotely doesn't publish salary at all.

Annualizing hourly pay needs a convention (2,080 hours/year is standard). Converting currencies needs live FX rates — and a stale rate silently corrupts your data in a way that's very hard to notice later.

3. The same job appears three times

Companies cross-post. If you merge three feeds naively, your "1,200 remote jobs" is closer to 1,100 real jobs plus a hundred duplicates, and any per-company counting is wrong.

Deduplicating needs a stable identity across boards. IDs and URLs are per-board, so the only usable key is company + title — and even that needs normalization, because "Acme, Inc." and "Acme" have to match while "Senior Engineer" and "Junior Engineer" must not.

Doing it yourself

Entirely possible. Here's the shape of it:

import httpx

def fetch_remoteok(client):
    payload = client.get("https://remoteok.com/api").json()
    for entry in payload:
        # The first element is a legal/metadata object, not a job.
        if entry.get("legal") is not None and not entry.get("position"):
            continue
        yield {
            "title": entry.get("position"),
            "company": entry.get("company"),
            # 0 means undisclosed, not free
            "salary_min": entry.get("salary_min") or None,
        }
Enter fullscreen mode Exit fullscreen mode

Budget for these gotchas, each of which cost me a debugging cycle:

  • RemoteOK's first array element is metadata, not a job. Treat it as one and you emit a junk record every run.
  • We Work Remotely returns 403 to obviously-automated user agents, even for its public RSS feed. A self-identifying UA silently loses the entire source — and if your logging isn't wired up, you won't notice you're missing a third of your data.
  • Himalayas' cursor can repeat. Follow it blindly and you loop forever.
  • Timestamps come in four formats: epoch seconds, epoch milliseconds, ISO-8601, and RFC-2822 from the RSS feed.

Then there's ongoing maintenance, because feeds change shape without notice.

Or use the ready-made one

I packaged all of the above as an Apify actor: Remote Jobs Feed.

It pulls all three boards, normalizes them into a single schema, annualizes salaries into a comparable USD field, and merges cross-postings — keeping the most complete copy and listing the other boards under also_on.

{
  "searchTerms": ["python", "backend"],
  "minSalaryUsd": 120000,
  "worldwideOnly": true,
  "maxResults": 50
}
Enter fullscreen mode Exit fullscreen mode

A typical run: 585 raw records in, 577 unique out after merging cross-postings, in about six seconds.

One deliberate choice worth calling out: non-USD salaries are not converted. The original figures are preserved and a salary_is_normalized flag tells you whether a number is comparable. Silently applying a stale exchange rate would look tidier and be worse.

Which to pick

Write it yourself if you need one board, or want total control over the schema. It's a day of work plus maintenance.

Use the actor if you want three boards reconciled and would rather not own the edge cases. It's $4 per 1,000 results, and the first run needs no configuration.

Either way, the thing to take from this post: the hard part isn't fetching, it's reconciling. Budget for that and you'll be fine.

Top comments (0)