DEV Community

Firsthand Data
Firsthand Data

Posted on Fully Autonomous

7 traps in job posting data (we hit all of them counting 61,600 jobs)

On October 21 we counted every open job at 2,871 tech companies, straight from their own career pages on Greenhouse, Lever, Ashby and SmartRecruiters: Hiring Pulse #1. The counting was fast. Making the numbers mean something was not.

Raw job data looks clean: a title, a location, a date, sometimes a salary. It isn't. Below are the seven traps we hit, each with a real example from our data and what we do about it. If you're building a job board, a hiring-signal feed for sales or a labor-market chart, you'll meet all seven.

1. One job, many rows

Companies post the same role once per city. In our snapshot, 61,600 postings collapsed to 54,226 unique company + title pairs: 12% of all rows were the same role in another location.

Sometimes it's more than a city. Tailscale's "Program Manager, EPD Operations" appeared twice on the same day: Remote (United States) at 114,000-143,000 USD and Remote (Canada) at 152,760-191,620 CAD. Same job, two rows, two currencies.

What to do: decide what you count. For "how many people is this company hiring", count company + title. For "where can I apply", keep the rows.

2. Not every job board is one employer

The second-busiest board in our snapshot, with 228 new postings in a week, belonged to Clera. It lists roles for many different startups, including 18 copies of "Founding Engineer", so it behaves like a recruiter, not like one company growing fast. Translation and AI-training talent pools do the same at scale: hundreds of postings like "Medical Translator - German into Bosnian".

What to do: flag boards whose titles repeat across unrelated teams or look like gig pools, and keep them out of "who is hiring" rankings.

3. Dates don't mean what you think

  • Batch days. Speechify published 246 roles on a single day (October 20), a third of its 738 open postings. Real growth or old roles re-posted? One snapshot can't tell.
  • Evergreen postings. Linear's "Senior / Staff Fullstack Engineer" was first published in April 2021 and was still open on October 20, 2026. Some companies keep a role open for years and hire from it again and again.
  • Weekends. New postings in one week: Saturday 165, Sunday 108, Monday 1,332, Tuesday 1,553. Measure on a Sunday and hiring looks frozen.

What to do: compare snapshots instead of trusting one date, and treat "new" as "first seen by you", not only as the published date.

4. "Remote" comes with fine print

"Remote (United States)" and "Remote (Canada)" are different jobs for a candidate in Berlin. "Remote, Bay Area, CA" means remote, but near San Francisco. Some hiring systems give a remote flag, others only the location text.

What to do: use the remote flag the company sets when there is one, keep the location text next to it and let people filter by both. In our snapshot 43% of postings were remote: 50% in data and AI, 38% in engineering.

5. A salary isn't a number until you parse it

About half of the postings (51%) published a pay range, but where and how varies:

  • Geography beats remote work. 69% of US postings show pay, Canada 66%, the UK 26%, India 9%. Remote and on-site postings show pay equally often (51% vs 52%).
  • Fields vs text. Some systems have salary fields; many companies put the range only in the description, in their own format.
  • Currency and period. USD, CAD, EUR and GBP in one feed; yearly and hourly ranges side by side. In Hiring Pulse #1 we treated only USD ranges of $30,000 or more as yearly, to keep hourly ones out of the medians.
  • Sales pay. Sales roles often post on-target earnings (base plus commission), which doesn't compare with an engineer's base salary.

What to do: store minimum, maximum, currency and period as separate fields, keep the original text, and never average across currencies or periods.

6. Locations are free text

Real location strings from one company's postings on October 21: "Poland, Serbia, Georgia, Cyprus" and "., Georgia". That's Georgia the country, not the US state, and a naive parser will put those jobs in Atlanta. Others from the same day: "North America", "Remote, USA", "Hybrid (Toronto, Ontario)".

What to do: parse to country and region where you can, keep every location of a multi-location job, and test your parser on the ambiguous names first.

7. Keyword filters match inside words

We group roles by words in the title, and short words bite. "ai" matches "paid", "llm" matches "fulfillment", "ux" matches "Linux", "uk" matches "Ukraine" and "us" matches "Austin". Our first filters did exactly that.

What to do: match whole words, keep a list of short words you never match inside other words, and read 20 random matches by eye before you publish a number.

Build it yourself or use a ready feed?

If you build it yourself, this is the list we ended up with:

  • four hiring systems, four APIs, four formats to map into one;
  • polite request rates (one of these APIs allows about 10 requests a second) and each site's rules;
  • a company list to maintain: companies switch systems, stop hiring or start;
  • duplicates, recruiter boards, dates, remote flags, salaries and locations from the traps above;
  • a memory of what you've already seen, so each run gives only the new jobs and tells you which ones closed.

None of it is hard alone. Together it's real work, and the maintenance never ends.

Disclosure: it's our product. If you'd rather get the new jobs from a list of companies every morning than build the pipeline, we run it on Apify and you can try it on the free plan: Job Monitor - New Job Alerts from Career Pages.

Hiring Pulse #2, with what changed in two weeks, comes out on November 4.

Written by an AI assistant (Claude); the data and every example are ours: a snapshot of 61,600 open jobs on October 21, 2026, and the daily runs of our job feeds.

Top comments (0)