Job data scraping is valuable for market research, salary analysis, and recruitment. Here's how to do it ethically.
Indeed Job Search
Indeed has a public search that returns results without login:
const cheerio = require('cheerio');
async function searchIndeed(query, location) {
const url = `https://www.indeed.com/jobs?q=${encodeURIComponent(query)}&l=${encodeURIComponent(location)}`;
const res = await fetch(url, {
headers: { 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36' }
});
const html = await res.text();
const $ = cheerio.load(html);
const jobs = [];
$('.job_seen_beacon, .resultContent').each((i, el) => {
jobs.push({
title: $(el).find('.jobTitle span').text().trim(),
company: $(el).find('.companyName').text().trim(),
location: $(el).find('.companyLocation').text().trim(),
snippet: $(el).find('.job-snippet').text().trim()
});
});
return jobs;
}
Better Alternative: Job Board APIs
Several job boards have free or cheap APIs:
| Platform | API | Free Tier |
|---|---|---|
| Indeed | Publisher API | Application required |
| Adzuna | Search API | 250 calls/day free |
| The Muse | Public API | Free, no key |
| GitHub Jobs | JSON endpoint | Free |
| RemoteOK | JSON feed | Free |
RemoteOK (Easiest — No Key)
async function getRemoteJobs() {
const res = await fetch('https://remoteok.com/api');
const jobs = await res.json();
return jobs.slice(1).map(j => ({
title: j.position,
company: j.company,
tags: j.tags,
salary: j.salary_min ? `$${j.salary_min}-${j.salary_max}` : 'Not specified',
url: j.url,
date: j.date
}));
}
Use Cases
- Salary research — what do web scrapers earn?
- Skill demand — which technologies are most requested?
- Market trends — which industries are hiring?
- Competitive intelligence — who are competitors hiring?
- HR analytics — time-to-fill, posting frequency
Resources
Need job market data extracted? Indeed, LinkedIn, Glassdoor — $20. Email: Spinov001@gmail.com | Hire me
Top comments (0)