Job postings data is messy. Every company uses a different ATS, and each ATS returns a completely different schema:

  • Greenhouse: integer IDs, location.name nested object, departments array
  • Lever: UUIDs, flat categories object, workplace type as separate field
  • Ashby: different ID format again, isRemote boolean, 970KB responses that stream-truncate

I built an Apify Actor to normalize all three into one schema. Technical notes:

The double-encoded HTML trap

Greenhouse serves some boards with double-encoded HTML: &lt;p&gt;Salary: $121,600...&lt;/p&gt; instead of <p>...</p>. One html.unescape() pass isn't enough — you need two, and you need to unescape before truncating, or a 400-char snippet becomes pure &lt; entities.

Salary extraction is a regex problem, not an AI problem

The biggest competitor in this space uses AI enrichment for salary fields — and has 0% coverage on the boards I tested. Plain regex with context windows ("salary range", "Salary:", "base pay", "Base pay: $85/hr") gets 37.8% overall and 77% on companies that publish ranges. Sometimes boring beats clever.

Rate limiting changes your architecture

My home IP got SSL-reset by one ATS after heavy polling. The fix: automatic proxy fallback (direct first, then proxy on connection errors). Without this, your nightly cron silently dies.

Incremental monitoring beats full downloads

Buyers don't want a 50k-row dataset re-downloaded daily. They want "alert me when tracked companies post new jobs". sinceDays + fingerprint dedup (company+title+location hash) turns a data dump into a monitoring service.

Live on the Apify Store: ATS Jobs Aggregator — normalized postings across 3 ATS platforms, salary extraction included.