DEV Community: Donglin Li

I aggregated 3 ATS job boards (Greenhouse, Lever, Ashby) into one normalized dataset — here's what I learned

Donglin Li — Thu, 12 Nov 2026 04:23:01 +0000

Job postings data is messy. Every company uses a different ATS, and each ATS returns a completely different schema:

Greenhouse: integer IDs, location.name nested object, departments array
Lever: UUIDs, flat categories object, workplace type as separate field
Ashby: different ID format again, isRemote boolean, 970KB responses that stream-truncate

I built an Apify Actor to normalize all three into one schema. Technical notes:

The double-encoded HTML trap

Greenhouse serves some boards with double-encoded HTML: <p>Salary: $121,600...</p> instead of <p>...</p>. One html.unescape() pass isn't enough — you need two, and you need to unescape before truncating, or a 400-char snippet becomes pure < entities.

Salary extraction is a regex problem, not an AI problem

The biggest competitor in this space uses AI enrichment for salary fields — and has 0% coverage on the boards I tested. Plain regex with context windows ("salary range", "Salary:", "base pay", "Base pay: $85/hr") gets 37.8% overall and 77% on companies that publish ranges. Sometimes boring beats clever.

Rate limiting changes your architecture

My home IP got SSL-reset by one ATS after heavy polling. The fix: automatic proxy fallback (direct first, then proxy on connection errors). Without this, your nightly cron silently dies.

Incremental monitoring beats full downloads

Buyers don't want a 50k-row dataset re-downloaded daily. They want "alert me when tracked companies post new jobs". sinceDays + fingerprint dedup (company+title+location hash) turns a data dump into a monitoring service.

Live on the Apify Store: ATS Jobs Aggregator — normalized postings across 3 ATS platforms, salary extraction included.

I aggregated 3 ATS job boards (Greenhouse, Lever, Ashby) into one normalized dataset — here's what I learned

Donglin Li — Thu, 12 Nov 2026 03:23:02 +0000

Job postings data is messy. Every company uses a different ATS, and each ATS returns a completely different schema:

Greenhouse: integer IDs, location.name nested object, departments array
Lever: UUIDs, flat categories object, workplace type as separate field
Ashby: different ID format again, isRemote boolean, 970KB responses that stream-truncate

I built an Apify Actor to normalize all three into one schema. Technical notes:

The double-encoded HTML trap

Salary extraction is a regex problem, not an AI problem

Rate limiting changes your architecture

My home IP got SSL-reset by one ATS after heavy polling. The fix: automatic proxy fallback (direct first, then proxy on connection errors). Without this, your nightly cron silently dies.

Incremental monitoring beats full downloads

Live on the Apify Store: ATS Jobs Aggregator — normalized postings across 3 ATS platforms, salary extraction included.

I aggregated 3 ATS job boards (Greenhouse, Lever, Ashby) into one normalized dataset — here's what I learned

Donglin Li — Thu, 12 Nov 2026 02:50:01 +0000

Job postings data is messy. Every company uses a different ATS, and each ATS returns a completely different schema:

Greenhouse: integer IDs, location.name nested object, departments array
Lever: UUIDs, flat categories object, workplace type as separate field
Ashby: different ID format again, isRemote boolean, 970KB responses that stream-truncate

I built an Apify Actor to normalize all three into one schema. Technical notes:

The double-encoded HTML trap

Salary extraction is a regex problem, not an AI problem

Rate limiting changes your architecture

My home IP got SSL-reset by one ATS after heavy polling. The fix: automatic proxy fallback (direct first, then proxy on connection errors). Without this, your nightly cron silently dies.

Incremental monitoring beats full downloads

Live on the Apify Store: ATS Jobs Aggregator — normalized postings across 3 ATS platforms, salary extraction included.

I aggregated 3 ATS job boards (Greenhouse, Lever, Ashby) into one normalized dataset — here's what I learned

Donglin Li — Thu, 12 Nov 2026 00:50:01 +0000

Job postings data is messy. Every company uses a different ATS, and each ATS returns a completely different schema:

Greenhouse: integer IDs, location.name nested object, departments array
Lever: UUIDs, flat categories object, workplace type as separate field
Ashby: different ID format again, isRemote boolean, 970KB responses that stream-truncate

I built an Apify Actor to normalize all three into one schema. Technical notes:

The double-encoded HTML trap

Salary extraction is a regex problem, not an AI problem

Rate limiting changes your architecture

My home IP got SSL-reset by one ATS after heavy polling. The fix: automatic proxy fallback (direct first, then proxy on connection errors). Without this, your nightly cron silently dies.

Incremental monitoring beats full downloads

Live on the Apify Store: ATS Jobs Aggregator — normalized postings across 3 ATS platforms, salary extraction included.

I aggregated 3 ATS job boards (Greenhouse, Lever, Ashby) into one normalized dataset — here's what I learned

Donglin Li — Wed, 11 Nov 2026 21:23:01 +0000

Job postings data is messy. Every company uses a different ATS, and each ATS returns a completely different schema:

Greenhouse: integer IDs, location.name nested object, departments array
Lever: UUIDs, flat categories object, workplace type as separate field
Ashby: different ID format again, isRemote boolean, 970KB responses that stream-truncate

I built an Apify Actor to normalize all three into one schema. Technical notes:

The double-encoded HTML trap

Salary extraction is a regex problem, not an AI problem

Rate limiting changes your architecture

My home IP got SSL-reset by one ATS after heavy polling. The fix: automatic proxy fallback (direct first, then proxy on connection errors). Without this, your nightly cron silently dies.

Incremental monitoring beats full downloads

Live on the Apify Store: ATS Jobs Aggregator — normalized postings across 3 ATS platforms, salary extraction included.

I aggregated 3 ATS job boards (Greenhouse, Lever, Ashby) into one normalized dataset — here's what I learned

Donglin Li — Wed, 11 Nov 2026 20:50:02 +0000

Job postings data is messy. Every company uses a different ATS, and each ATS returns a completely different schema:

Greenhouse: integer IDs, location.name nested object, departments array
Lever: UUIDs, flat categories object, workplace type as separate field
Ashby: different ID format again, isRemote boolean, 970KB responses that stream-truncate

I built an Apify Actor to normalize all three into one schema. Technical notes:

The double-encoded HTML trap

Salary extraction is a regex problem, not an AI problem

Rate limiting changes your architecture

My home IP got SSL-reset by one ATS after heavy polling. The fix: automatic proxy fallback (direct first, then proxy on connection errors). Without this, your nightly cron silently dies.

Incremental monitoring beats full downloads

Live on the Apify Store: ATS Jobs Aggregator — normalized postings across 3 ATS platforms, salary extraction included.

I aggregated 3 ATS job boards (Greenhouse, Lever, Ashby) into one normalized dataset — here's what I learned

Donglin Li — Wed, 11 Nov 2026 19:50:02 +0000

Job postings data is messy. Every company uses a different ATS, and each ATS returns a completely different schema:

Greenhouse: integer IDs, location.name nested object, departments array
Lever: UUIDs, flat categories object, workplace type as separate field
Ashby: different ID format again, isRemote boolean, 970KB responses that stream-truncate

I built an Apify Actor to normalize all three into one schema. Technical notes:

The double-encoded HTML trap

Salary extraction is a regex problem, not an AI problem

Rate limiting changes your architecture

My home IP got SSL-reset by one ATS after heavy polling. The fix: automatic proxy fallback (direct first, then proxy on connection errors). Without this, your nightly cron silently dies.

Incremental monitoring beats full downloads

Live on the Apify Store: ATS Jobs Aggregator — normalized postings across 3 ATS platforms, salary extraction included.

I aggregated 3 ATS job boards (Greenhouse, Lever, Ashby) into one normalized dataset — here's what I learned

Donglin Li — Wed, 11 Nov 2026 17:50:02 +0000

Job postings data is messy. Every company uses a different ATS, and each ATS returns a completely different schema:

Greenhouse: integer IDs, location.name nested object, departments array
Lever: UUIDs, flat categories object, workplace type as separate field
Ashby: different ID format again, isRemote boolean, 970KB responses that stream-truncate

I built an Apify Actor to normalize all three into one schema. Technical notes:

The double-encoded HTML trap

Salary extraction is a regex problem, not an AI problem

Rate limiting changes your architecture

My home IP got SSL-reset by one ATS after heavy polling. The fix: automatic proxy fallback (direct first, then proxy on connection errors). Without this, your nightly cron silently dies.

Incremental monitoring beats full downloads

Live on the Apify Store: ATS Jobs Aggregator — normalized postings across 3 ATS platforms, salary extraction included.

I aggregated 3 ATS job boards (Greenhouse, Lever, Ashby) into one normalized dataset — here's what I learned

Donglin Li — Wed, 11 Nov 2026 16:14:53 +0000

Job postings data is messy. Every company uses a different ATS, and each ATS returns a completely different schema:

Greenhouse: integer IDs, location.name nested object, departments array
Lever: UUIDs, flat categories object, workplace type as separate field
Ashby: different ID format again, isRemote boolean, 970KB responses that stream-truncate

I built an Apify Actor to normalize all three into one schema. Technical notes:

The double-encoded HTML trap

Salary extraction is a regex problem, not an AI problem

Rate limiting changes your architecture

My home IP got SSL-reset by one ATS after heavy polling. The fix: automatic proxy fallback (direct first, then proxy on connection errors). Without this, your nightly cron silently dies.

Incremental monitoring beats full downloads

Live on the Apify Store: ATS Jobs Aggregator — normalized postings across 3 ATS platforms, salary extraction included.