DEV Community

Rishabh Singh
Rishabh Singh

Posted on

One Workflow to Scrape ANY Job Board or Career Page of any company

n8n and Bright Challenge: Unstoppable Workflow

Searching for jobs can feel like a full-time job in itself.
Scrolling through listings, filtering by keywords, tailoring applications, it eats up hours every week.

So I decided to do what any lazy developer would do... I automated it. 🚀

Just add a URL in sheet, workflow will find, filter and notify.

Watch Demo 🎥

The Problem

  • Too many platforms (LinkedIn, Indeed, remote job boards, etc.)
  • Same repetitive filtering process every day
  • Hard to keep track of new postings

Instead of wasting my energy on job hunting, I wanted a workflow that does the boring stuff for me, only shows me relevant opportunities, and notifies me instantly.

The Solution: n8n + BrightData

If you haven’t used it before, n8n is an open-source workflow automation tool.

Here’s how my setup works:

  1. Visit the URLs of career pages or job boards, use the filters available on the site and add the URL into a Google Sheet.
  2. The workflow fetches openings from those URLs, and includes/excludes jobs if the role title contains certain keywords.
  3. For each matching job, the workflow scrapes the full job description.
  4. An AI model analyzes the description and decides if the job is actually relevant.
  5. If it passes the check, I get a Telegram notification instantly.

Workflow

Result:

No more endless scrolling, just the jobs that actually matter to me.

Result

Best Part:

Since, extraction works by extracting links and their link texts, then visiting those links, so this workflow will work on any job board with two step filter(scraper + AI)!. No need of customizing the workflow for LinkedIn, Indeed, lever, workday...

Why This Is a Game Changer

  • Saves hours of job hunting every week
  • Never miss new opportunities
  • Scales easily, I can add more job sources anytime
  • Feels like I have a personal job-hunting assistant

Gist

https://gist.github.com/Heapnotizer/82153323498bdf95765ce8232d0a960a

💬 Any Suggestions? I’d love to hear your thoughts in the comments!

Top comments (2)

Collapse
 
diegomtz profile image
Diego Martinez

Love this! What's next—will you publish the n8n template/repo and add things like deduping, resume-matching, or auto-apply? Also curious if you'll share notes on handling anti-bot pages and rate limits.

Collapse
 
heapnotizer profile image
Rishabh Singh

The gist has n8n template. But you will have to make scraper on your own, just extract the links!

Resume Matching: Can be handled by passing details to AI agent node.
Auto apply: Working on that. Though captcha's usually occur while applying.
Deduping: I guess we can store the scraped openings in a sheet and next time the scraped links match, we skip.