DEV Community

Cedarfetch Labs
Cedarfetch Labs

Posted on AI-assisted

How to build a daily "new remote jobs" feed from public job-board APIs

You want a daily list of new remote jobs for yourself, a newsletter or a small job board. Several boards publish a JSON API or an RSS feed for exactly this. Each one comes with attribution rules and polling limits, though, and the same job often shows up on two boards.

Boards with open feeds

I called two of these for this post and read the terms of the rest on 2026-11-11. Re-read each board's page before you ship anything.

  • Remote OK: https://remoteok.com/api is one JSON request. Element 0 is a legal notice, then the ~100 most recent jobs (98 today). The terms: link back to the job with a followed link (no nofollow), name Remote OK as the source, and do not use its logo.
  • We Work Remotely: RSS per category, for example https://weworkremotely.com/categories/remote-programming-jobs.rss (25 items today). Anyone may use the feeds; attribute the links back to the board.
  • Remotive: https://remotive.com/api/remote-jobs, published with a 24-hour delay and meant to be polled about four times a day at most. Link back, mention Remotive, and do not forward its jobs to Google Jobs, LinkedIn Jobs, Jooble or Neuvoo.
  • Jobicy: https://jobicy.com/api/jobs?count=100, the last 7 days, no more than once an hour. Credit Jobicy and keep apply links on its job page.
  • Others: 4dayweek.io allows 60 requests a minute and asks for a credit; Remote First Jobs has the same "do not forward" rule as Remotive; Arbeitnow is a free API for mostly German jobs, "do not abuse".

Terms differ enough to matter. At least one board with an open API, Himalayas, forbids passing its jobs on to third parties, so I left it out.

Fetch two boards

import json, re, urllib.request
import xml.etree.ElementTree as ET

UA = {"User-Agent": "remote-feed-example/1.0 (personal script)"}

def fetch(url):
    with urllib.request.urlopen(urllib.request.Request(url, headers=UA), timeout=30) as r:
        return r.read()

def remoteok():
    for j in json.loads(fetch("https://remoteok.com/api"))[1:]:   # element 0 is the legal notice
        yield dict(board="remoteok", id=j["id"], company=j["company"], title=j["position"],
                   url=j["url"], posted=j["date"])

def weworkremotely():
    root = ET.fromstring(fetch("https://weworkremotely.com/categories/remote-programming-jobs.rss"))
    for it in root.iter("item"):
        company, _, title = it.findtext("title").partition(": ")  # "Company: Title"
        yield dict(board="weworkremotely", id=it.findtext("guid"), company=company, title=title,
                   url=it.findtext("link"), posted=it.findtext("pubDate"))
Enter fullscreen mode Exit fullscreen mode

Output:

remoteok 98 jobs; first: Mento | Strategy & Operations Coach APAC | 2026-11-11T00:00:15+00:00
weworkremotely 25 jobs; first: BDF EXPERTS | SAP Berater FI/CO (M/W/D) | Mon, 19 Oct 2026 10:01:21 +0000
Enter fullscreen mode Exit fullscreen mode

We Work Remotely puts "Company: Title" in one field, so the script splits on the first ": ". Dates arrive in different formats; normalise them before you sort.

Deduplicate and keep only what is new

Save the first script as boards.py. This one builds a key from company and title, merges matches across boards, and remembers keys between runs.

import json, os, re
from boards import remoteok, weworkremotely

def key(job):
    norm = lambda s: re.sub(r"[^a-z0-9]+", " ", s.lower()).strip()
    title = re.sub(r"\b(remote|m f d|m w d)\b", "", norm(job["title"])).strip()
    return norm(job["company"]) + "|" + title

clusters = {}   # key -> clusters; a cluster never holds two jobs from the same board
for job in [*remoteok(), *weworkremotely()]:
    for cluster in clusters.setdefault(key(job), []):
        if all(j["board"] != job["board"] for j in cluster):
            cluster.append(job)
            break
    else:
        clusters[key(job)].append([job])

seen = set(json.load(open("seen.json"))) if os.path.exists("seen.json") else set()
for k, lists in clusters.items():
    if k not in seen:
        for c in lists:
            boards = "+".join(j["board"] for j in c)
            print(f'{boards:28} {c[0]["company"]} | {c[0]["title"]} | {c[0]["url"]}')
json.dump(sorted(clusters), open("seen.json", "w"))
Enter fullscreen mode Exit fullscreen mode

The first run printed all 122 postings. I then removed five keys from seen.json and ran it again, so those five came out as new:

remoteok+weworkremotely      Sanctuary Computer | Head of Technology | https://remoteOK.com/remote-jobs/remote-head-of-technology-sanctuary-computer-1137537
remoteok                     Addepto | Data Engineer | https://remoteOK.com/remote-jobs/remote-data-engineer-addepto-1137714
weworkremotely               Airbnb | Operations Engineer, BizTech | https://weworkremotely.com/remote-jobs/airbnb-operations-engineer-biztech
Enter fullscreen mode Exit fullscreen mode

The first line is a real cross-post. Two different listings from the same board with the same company and title also turned up, and the script keeps those apart: two postings on one board are usually separate openings, one per region for example. Only a match across different boards is treated as a duplicate.

Limits and gotchas

  • Attribution is part of the deal. Show the source board and link to its job URL on every item. Remote OK returns URLs with a capitalised host (remoteOK.com); lowercase it.
  • Messy data. Remote OK text can arrive double-encoded (München for München), its location is often an office city or empty, and I saw salary_min of 0 as a placeholder. Treat salaries as hints.
  • "New" means new to the feed. Some feeds are delayed by hours or a day, and some only show a recent window, so a job that drops out of the window has not necessarily closed.
  • Respect the windows. Cache what you fetch and poll no faster than each board says. Do not republish a board's jobs somewhere its terms forbid.
  • No workarounds. Boards behind bot protection or without a public feed are out of scope; one board answered 403 to a plain request while I was building the Actor. Move on instead of fighting it.

If you don't want to maintain this yourself

Disclosure: I'm the developer of an Apify Actor, Remote Jobs Scraper, published under Cedarfetch Labs. It reads seven boards (Remote OK, We Work Remotely, Remotive, Jobicy, 4dayweek.io, Remote First Jobs and Arbeitnow) through their official feeds and returns one schema with the location restriction parsed into countries and regions. It merges cross-posts and lists the other boards in alsoOn, puts each board's attribution terms on every row, and with change tracking on a daily schedule it can return only new jobs.

It is pay per event: $0.0015 per job row plus $0.00005 per run start, as shown on the listing on 2026-11-11. Check the Pricing tab for current figures. Listing: https://apify.com/cedarfetch/remote-jobs-scraper

Runnable Python, Node.js and cURL examples for the Actor: https://github.com/cedarfetch/remote-jobs-scraper-examples

Not affiliated with any of the boards mentioned.

Top comments (0)