Disclosure: this uses Career Site Jobs, a tool I built. The approach works with any source that gives you new and closed jobs.
Every niche job board hits the same wall: employers don't post enough paid listings at first, so the board looks empty, so job seekers don't come back, so employers don't post. The usual fix is backfill: fill the board with real jobs from elsewhere until paid listings take over.
The catch is that most backfill comes from big aggregators, which means duplicates, agency reposts and jobs that were filled weeks ago. Here's how to backfill straight from employers' own career sites instead, in about 40 lines of Python, with filled jobs removed automatically.
Why career sites
Companies post jobs on their own careers page first: Greenhouse, Workday, Lever, Ashby, iCIMS and the rest. That page links to the real apply form, and the job disappears from it when the role is filled. So if you mirror career sites, your board stays as fresh as the employers' own pages.
Step 1: describe your niche as a search
Say your board is "remote data jobs in the US." As a search, that's:
{ "keywords": ["data engineer", "data analyst", "analytics engineer"], "countries": ["US"], "workplaceTypes": ["remote"], "onlyNewSinceLastRun": true, "includeClosedJobs": true, "feedName": "my-board", "maxResults": 1000 } -
keywordsmatch job titles (a job matches if its title has all the words of any keyword). -
onlyNewSinceLastRunmakes each run return only jobs that appeared since your previous run. -
includeClosedJobsadds the jobs employers took down since then, marked"status": "closed". -
feedNamekeeps your place even if you tweak the filters later.
Try it once in the browser first: the daily feed search has this set up. The first run returns everything that matches today, which is your initial backfill.
Step 2: sync it into your board every day
Install the client: pip install apify-client. Then:
import os from apify_client import ApifyClient client = ApifyClient(os.environ["APIFY_TOKEN"]) run_input = { "keywords": ["data engineer", "data analyst", "analytics engineer"], "countries": ["US"], "workplaceTypes": ["remote"], "onlyNewSinceLastRun": True, "includeClosedJobs": True, "feedName": "my-board", "maxResults": 1000, } run = client.actor("trap_check/career-site-jobs").call(run_input=run_input) added = removed = 0 for job in client.dataset(run["defaultDatasetId"]).iterate_items(): if job.get("status") == "closed": remove_from_board(job["id"]) # your code: unpublish it removed += 1 else: upsert_on_board({ # your code: insert or update by id "id": job["id"], # stable, use it as your key "title": job["title"], "company": job["company"], "location": job["location"], "remote": job.get("workplaceType"), "salary": job.get("salary"), # e.g. "$120k–150k / yr", when posted "posted_at": job.get("postedAt"), "apply_url": job["url"], # the employer's own apply page }) added += 1 print(f"added {added}, removed {removed}") remove_from_board and upsert_on_board are wherever your board keeps jobs: a database table, a WordPress job plugin's API, Airtable, a JSON file your static site builds from.
Step 3: run it daily
Pick one:
- Cron or GitHub Actions: run the script once a day.
- No code: in Apify, save the search as a task, add a daily schedule, and connect the results to Zapier, Make or a webhook that posts into your board.
What it costs
$3 per 1,000 jobs returned, and $1 per 1,000 closed jobs. A niche board pulling 100 new jobs a day pays about $9 a month. Apify's free monthly credit covers about 1,600 jobs, which is enough to test.
Tips from running this
- Start narrow. A board with 300 relevant jobs beats one with 30,000 random ones. Use
excludeKeywords(e.g. "senior", "sales") to trim. - Label backfilled jobs. Many boards show them with a small "via company site" tag and rank paid listings above them.
- Keep the apply link pointing to the employer. It's what job seekers want, and it's what makes backfill legitimate.
- Check coverage for your niche first. Search a few employers you'd expect on your board by name, using the
companiesfield.
If you'd rather look at the data before writing code, here's a free CSV of 31,752 jobs posted in one day.
![AMR Mechanical and Electrical Engineering — Ch.12: Contents {#contents .TOC-Heading} [KR]](https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fwzu7uvk9b9zcljsl3gz5.png)
![AMR Mechanical and Electrical Engineering — Ch.14: Contents {#contents .TOC-Heading} [KR]](https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F6nel7ut2runbhcvsezvf.png)
