TL;DR: You do not need to refresh five job boards every morning. There are three ways to get every new posting into one place. First, turn on the saved search email alerts inside each board and funnel them into one inbox folder. Second, poll the official APIs of USAJobs and arbeitnow with a short Python script that remembers what it has already seen and prints only what is new. Third, run a scheduled monitor that does the remembering for you. Delta Tools built one for exactly this job. It is called the Job Board New Posting Monitor, and it watches USAJobs and arbeitnow searches and reports only genuinely new postings. All three methods are below, with working code for the second one.
I spent one job hunt doing this the stupid way. Five boards. Five tabs. Every morning I ran the same searches, tried to remember which titles I had already seen, and told myself I was being thorough. I was not being thorough. I was being slow. By the time I found one posting I liked, it had been live for four days, and anyone who applied on day one was already ahead of me in the queue.
Job hunting rewards speed more than almost anyone admits. Recruiters start reading applications while a posting is still gathering them. Some roles close within a week. The edge is not a better resume template. The edge is simply knowing a role exists on the morning it appears, not the morning you happen to check.
Here are three ways to get that edge, from simplest to fully automated.
Before we start: the Delta Tools shortcut for all of this is our Job Board New Posting Monitor, which watches up to 20 searches at once on a schedule and reports only postings it has never shown you before. The rest of this article teaches you to build the same thing by hand, in case you would rather run your own setup.
Method 1: Turn on the alerts each board already gives you (15 minutes, free)
Almost every large board has the same feature hiding under a slightly different name. You run a search, you save it, and the board emails you when new postings match. Most job seekers never turn it on. They bookmark the search results page instead, which is the digital equivalent of walking to the shop every day to see if the bread arrived.
USAJobs is the clearest example, and it is worth doing properly because federal roles have hard closing dates. Sign in, run your search with your keywords and location, and click Save this search above the results. Give it a name and choose how often you want email. Daily is the right choice for anything competitive. USAJobs lets you keep up to 10 saved searches at once, and a saved search expires after one year, so put a note in your calendar to renew the ones that still matter. The notification emails come from notifications@usajobs.gov, and they only list jobs posted since your last email, which means the board is doing the remembering for you on its own turf.
LinkedIn and Indeed offer the same pattern. Run the search, set the filters, and switch on the email alert. The names of the buttons differ. The idea does not.
Now for the part that turns five alert streams into one feed. Create one folder or label in your inbox called Job Alerts. Add a filter rule for each sender address so every alert lands in that folder instead of scattering across your inbox. USAJobs sends from the address above. Indeed and LinkedIn send from their own alert addresses, and you can capture each one from the first email it sends you. From then on, your morning routine is one folder, read once, in date order. Everything new is in there. Nothing requires you to remember what you saw yesterday.
When this is enough: you follow a handful of searches, the boards you care about all have decent alert emails, and you actually read email every morning.
When it is not: the board you need has weak alerts or none at all, you want alerts inside Slack or Discord instead of email, or you follow enough searches that the daily folder becomes its own chore. Keep reading.
Method 2: Poll the official APIs with a script (free, one feed you control)
Two of the most useful boards publish official APIs, and they could not be more different in spirit.
USAJobs runs the official jobs board of the United States federal government. Its API at data.usajobs.gov is free, but it wants a personal API key. You request one on the site, and every request sends that key in a header called Authorization-Key, plus a User-Agent header set to your email address. In return you get structured data with a gift inside it: every posting carries a stable PositionID that never changes for the life of that posting.
arbeitnow is a European board with a large share of English language postings, many of them open to remote work. Its job board API is free and needs no key at all. One GET request to its public endpoint returns a batch of recent postings as JSON, wrapped in a field called data, and you page through older batches by adding a page parameter. It has no search parameter on the server side, so filtering happens on your own machine: you fetch a few pages and keep the postings that match your keyword. Every posting carries a slug, the unique tail of its URL, and that slug is its stable identity.
Stable identity is the whole game. Here is why. You cannot detect new postings by watching the top of a list, because lists move. A posting that sat at position three yesterday sits at position nine today, and a board may edit a title or extend a closing date and push an old posting back into view. If your script compares positions or titles, it will cry wolf constantly, and you will stop reading its alerts within a week. If it compares stable IDs against a set of IDs it has already reported, it stays silent until something genuinely new appears. Dedup is the whole product. Everything else is plumbing.
Here is the whole thing in one script. It checks three pages of arbeitnow and one USAJobs search, prints every posting it has never seen before, and optionally posts each one to a webhook so the alert lands in Slack or Discord or anywhere else that accepts a JSON post.
import json
import os
import urllib.parse
import urllib.request
from datetime import datetime, timezone
SEEN_FILE = "seen_jobs.json"
WEBHOOK_URL = os.environ.get("JOB_WEBHOOK_URL", "")
USAJOBS_KEY = os.environ.get("USAJOBS_API_KEY", "")
USAJOBS_EMAIL = os.environ.get("USAJOBS_EMAIL", "")
ARBEITNOW_API = "https://www.arbeitnow.com/api/job-board-api"
USAJOBS_API = "https://data.usajobs.gov/api/search"
def get_json(url, headers=None):
req = urllib.request.Request(url, headers=headers or {})
with urllib.request.urlopen(req, timeout=45) as resp:
return json.load(resp)
def fetch_arbeitnow(keyword, pages=3):
found = []
needle = keyword.lower()
for page in range(1, pages + 1):
data = get_json(f"{ARBEITNOW_API}?page={page}")
items = data.get("data") or []
if not items:
break
for job in items:
haystack = " ".join([
job.get("title") or "",
job.get("company_name") or "",
" ".join(job.get("tags") or []),
]).lower()
if needle in haystack:
found.append({
"id": "arbeitnow:" + str(job.get("slug") or ""),
"title": job.get("title") or "",
"company": job.get("company_name") or "",
"location": job.get("location") or "",
"url": job.get("url") or "",
})
return found
def fetch_usajobs(keyword):
if not USAJOBS_KEY:
print("USAJobs skipped: no API key set.")
return []
params = {
"Keyword": keyword,
"ResultsPerPage": "25",
"Page": "1",
}
url = USAJOBS_API + "?" + urllib.parse.urlencode(params)
headers = {
"Host": "data.usajobs.gov",
"Authorization-Key": USAJOBS_KEY,
"User-Agent": USAJOBS_EMAIL,
}
data = get_json(url, headers=headers)
found = []
for item in data.get("SearchResultItems") or []:
d = item.get("MatchedObjectDescriptor") or {}
job_id = d.get("PositionID") or item.get("MatchedObjectId")
if not job_id:
continue
found.append({
"id": "usajobs:" + str(job_id),
"title": d.get("PositionTitle") or "",
"company": d.get("OrganizationName") or "",
"location": d.get("PositionLocationDisplay") or "",
"url": d.get("PositionURI") or "",
})
return found
def announce(job):
line = f"NEW: {job['title']} at {job['company']} ({job['location']})\n {job['url']}"
print(line)
if WEBHOOK_URL:
payload = json.dumps({"text": line}).encode()
req = urllib.request.Request(
WEBHOOK_URL, data=payload,
headers={"Content-Type": "application/json"},
)
urllib.request.urlopen(req, timeout=30).read()
def main():
first_run = not os.path.exists(SEEN_FILE)
seen = set()
if not first_run:
seen = set(json.load(open(SEEN_FILE)))
jobs = fetch_arbeitnow("software engineer") + fetch_usajobs("data analyst")
new_jobs = [j for j in jobs if j["id"] not in seen]
for job in new_jobs:
if not first_run:
announce(job)
seen.update(j["id"] for j in jobs)
json.dump(sorted(seen), open(SEEN_FILE, "w"))
stamp = datetime.now(timezone.utc).isoformat(timespec="seconds")
if first_run:
print(f"{stamp} Baseline built: {len(jobs)} postings recorded, none announced.")
print("Run it again tomorrow and only genuinely new postings will print.")
else:
print(f"{stamp} Checked {len(jobs)} postings, {len(new_jobs)} new.")
if __name__ == "__main__":
main()
Three details in that script deserve a callout, because each one cost me a round of duplicate alerts before I learned it.
The first run builds a baseline in silence. If the script announced everything on its first run, you would get a flood of postings that have been sitting there for weeks, and half of them would already be closed to serious competition. So on the first run it records what exists and announces nothing. From the second run onward, it announces only IDs it has never recorded. If you would rather see the full list once, announce on the first run too. Just know what you are asking for.
The seen file is the entire database, and it is fragile in specific ways. It lives on the machine that runs the script. If that machine is asleep at the scheduled hour, that hour simply never happens. If the file gets deleted, the baseline resets and the flood returns. If the script crashes after announcing a posting but before saving the file, you will hear about that posting twice. None of this is hard to fix. All of it is yours to fix, forever, at 11 p.m. on a Sunday.
Keyword matching on arbeitnow is honest but blunt. Because the API has no server side search, the script fetches recent pages and matches your keyword against the title, the company name, and the tags. Postings that describe your role with a synonym you did not pick will slip past. Run two or three keyword variants if the role has more than one common name. And keep the page count modest. arbeitnow gives this API away for free and asks callers not to abuse it, and three pages every hour is well inside polite territory.
Schedule it with cron on Linux or macOS, or with Task Scheduler on Windows, every hour or two during the hours you care about. Set the JOB_WEBHOOK_URL environment variable to a Slack or Discord webhook and the prints become push alerts in a channel you already read. Set USAJOBS_API_KEY and USAJOBS_EMAIL and the federal postings join the same feed. One feed. One place to look.
If you want the middle ground between Methods 1 and 2, this is also where I should mention the Delta Tools option again, because it exists for the exact moment the script starts feeling like a second job: the Job Board New Posting Monitor runs this same pattern on a schedule, with the remembering handled for you.
Method 3: A scheduled monitor that handles the boring parts (no infrastructure)
Method 3 is Method 2 with the fragile parts removed. The monitor is a small Apify actor. You give it your searches, up to 20 in a single run, and each search names a source, USAJobs or arbeitnow, plus a keyword and its own options. USAJobs searches accept a location and a freshness window, so your first run can ignore postings older than seven days instead of drowning in history. arbeitnow searches accept a location filter and a remote only switch. You paste in your free USAJobs key if you use federal searches. arbeitnow needs no key at all, and if you leave the key out, the USAJobs searches skip gracefully while the arbeitnow searches still run.
On every scheduled run it fetches the current postings, compares stable IDs against a snapshot it keeps between runs, and writes only the genuinely new ones to its output dataset. Each row carries the title, the company, the location, the posting date, and a direct link, tagged with the search that found it. It can also report postings that disappeared, which matters more than it sounds: a posting vanishing inside its window usually means the role was filled or pulled, and knowing that stops you polishing an application for a door that already closed. Rows about removed postings are never billed.
The pricing matches the philosophy. You pay $0.001 when a run starts and $0.0015 for each genuinely new posting it delivers. A scheduled run that finds nothing new costs the start fee and nothing more. Compare that with scrapers that charge for every row on every run, including the ninety percent of rows you saw yesterday. On this design, a quiet week costs you almost nothing, and a busy week costs you pennies in exchange for being early.
The output dataset plugs into the rest of your life through Apify integrations and webhooks, so new postings can land in Slack, in email, or in a Google Sheet you share with a friend who is hunting for the same roles. Give each scheduled monitor its own stateKey if you run more than one, and their memories stay separate. One key for your own search, another for the contract roles you track for a client, no cross talk between them.
I will not pretend the script in Method 2 is a lesser choice. It is free, it is transparent, and the code above is yours to keep and change. Plenty of people should run the script. The actor exists for people who have noticed that their laptop sleeps, their cron file does not survive a new machine, and their seen file is one careless cleanup away from a Monday morning flood. The Job Board New Posting Monitor is the version where someone else keeps the lights on, and you just read the feed!
FAQ
How often should I check for new postings?
Hourly during waking hours is plenty for most roles. Federal postings on USAJobs usually stay open for days or weeks, so even a daily check beats the crowd there. Competitive commercial roles can gather a full slate of applicants fast, and for those, hourly checks on a schedule beat a daily email digest.
Can I watch more than one board in a single feed?
Yes, and that is the point of this article. Method 1 funnels the alert emails of several boards into one inbox folder. Method 2 merges arbeitnow and USAJobs into one printed or webhooked feed. Method 3 runs up to 20 searches across both sources in a single scheduled run and merges the results into one dataset.
Why not just scrape LinkedIn or Indeed?
Because you do not need to, and the ground there is genuinely hostile. Both platforms work hard to block automated collection, and accounts get restricted for it. The boards in this article publish official APIs that hand you the same postings in structured form, with stable IDs made for exactly this job. Use the door the owner built.
What does new actually mean in these methods?
It means an ID your system has never recorded before. Not a posting near the top of a list. Not a posting with an edited title. A stable posting ID, a PositionID on USAJobs or a slug on arbeitnow, appearing for the first time in your snapshot. That definition is what keeps the feed quiet on ordinary days and loud only when it should be.
How much does the monitor cost on a quiet week?
Almost nothing, by design. Each run costs $0.001 to start, and new postings cost $0.0015 each. A daily schedule on a week where nothing matches costs under a cent in start fees. You pay in proportion to how much genuinely new material shows up, which is how monitoring ought to be priced.
Do I need an API key to start?
Only for USAJobs, and that key is free from data.usajobs.gov. arbeitnow needs no key, no account, and no setup, so you can run the script in Method 2 against arbeitnow in the next five minutes and add the federal feed whenever you are ready.
Top comments (0)