DEV Community

Алексей Спинов
Алексей Спинов

Posted on

How to Scrape Job Listings from Indeed and LinkedIn Jobs

Job data scraping is valuable for market research, salary analysis, and recruitment. Here's how to do it ethically.

Indeed Job Search

Indeed has a public search that returns results without login:

const cheerio = require('cheerio');

async function searchIndeed(query, location) {
  const url = `https://www.indeed.com/jobs?q=${encodeURIComponent(query)}&l=${encodeURIComponent(location)}`;
  const res = await fetch(url, {
    headers: { 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36' }
  });
  const html = await res.text();
  const $ = cheerio.load(html);

  const jobs = [];
  $('.job_seen_beacon, .resultContent').each((i, el) => {
    jobs.push({
      title: $(el).find('.jobTitle span').text().trim(),
      company: $(el).find('.companyName').text().trim(),
      location: $(el).find('.companyLocation').text().trim(),
      snippet: $(el).find('.job-snippet').text().trim()
    });
  });
  return jobs;
}
Enter fullscreen mode Exit fullscreen mode

Better Alternative: Job Board APIs

Several job boards have free or cheap APIs:

Platform API Free Tier
Indeed Publisher API Application required
Adzuna Search API 250 calls/day free
The Muse Public API Free, no key
GitHub Jobs JSON endpoint Free
RemoteOK JSON feed Free

RemoteOK (Easiest — No Key)

async function getRemoteJobs() {
  const res = await fetch('https://remoteok.com/api');
  const jobs = await res.json();
  return jobs.slice(1).map(j => ({
    title: j.position,
    company: j.company,
    tags: j.tags,
    salary: j.salary_min ? `$${j.salary_min}-${j.salary_max}` : 'Not specified',
    url: j.url,
    date: j.date
  }));
}
Enter fullscreen mode Exit fullscreen mode

Use Cases

  1. Salary research — what do web scrapers earn?
  2. Skill demand — which technologies are most requested?
  3. Market trends — which industries are hiring?
  4. Competitive intelligence — who are competitors hiring?
  5. HR analytics — time-to-fill, posting frequency

Resources


Need job market data extracted? Indeed, LinkedIn, Glassdoor — $20. Email: Spinov001@gmail.com | Hire me

Top comments (0)