DEV Community

agenthustler
agenthustler

Posted on

Best LinkedIn Job Scrapers in 2026: Public Job Data via Apify

LinkedIn is the largest professional job board in the world. What many people don't realize is that LinkedIn exposes job listings through a public endpoint that requires no login — the jobs-guest API. This makes LinkedIn job data accessible for market research, recruiting analytics, and salary analysis without violating account terms.

The LinkedIn Jobs-Guest Endpoint

LinkedIn serves public job listings at URLs like:

https://www.linkedin.com/jobs-guest/jobs/api/seeMoreJobPostings/search
    ?keywords=python+developer
    &location=United+States
    &start=0
Enter fullscreen mode Exit fullscreen mode

This endpoint returns HTML fragments with job cards. No authentication needed. It's the same data LinkedIn shows to logged-out visitors when they browse job listings.

What you can extract:

  • Job title, company name, location
  • Posted date, applicant count
  • Job description (from individual job pages)
  • Employment type (full-time, contract, etc.)
  • Seniority level
  • Industry and function tags

Why Scrape LinkedIn Jobs?

The use cases go well beyond job hunting:

  • Job market research — track demand for specific skills across regions and industries
  • Recruiting intelligence — identify which companies are hiring aggressively, what roles they're filling
  • Salary analysis — correlate job descriptions with salary ranges (when disclosed)
  • Competitive analysis — monitor competitor hiring patterns to infer strategic direction
  • Academic research — labor economics, skill demand trends, geographic employment patterns
  • Career tools — build job aggregators, alert services, or recommendation engines

Approach 1: Apify Store Actors

The Apify Store has several LinkedIn job scrapers. These cloud-based actors handle proxy rotation and pagination automatically.

Key considerations when choosing an actor:

  • Uses jobs-guest endpoint — actors that require LinkedIn login are riskier and may violate LinkedIn's Terms of Service
  • Pagination support — LinkedIn caps results at 1000 per search; good actors work around this by narrowing search parameters
  • Data freshness — check when the actor was last updated; LinkedIn changes its HTML frequently
  • Output fields — some actors only grab titles and links; better ones extract full job descriptions

Popular actors include general-purpose LinkedIn scrapers as well as job-specific ones. Search "linkedin jobs" on the Apify Store to see current options and their ratings.

Our Upcoming Actor

We're building a focused LinkedIn Jobs scraper at apify.com/cryptosignals/linkedin-jobs-scraper designed around the public jobs-guest endpoint:

  • No login required — uses only publicly accessible data
  • Full job details — title, company, location, description, seniority, employment type
  • Smart pagination — splits broad searches into narrower queries to bypass the 1000-result cap
  • Structured output — clean JSON with consistent field names

This actor is upcoming and not yet publicly available — check the link for launch updates.

Approach 2: DIY with Python

Here's a basic scraper using the jobs-guest endpoint:

import requests
from bs4 import BeautifulSoup
import time

HEADERS = {
    "User-Agent": (
        "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) "
        "AppleWebKit/537.36 (KHTML, like Gecko) "
        "Chrome/120.0.0.0 Safari/537.36"
    )
}

def search_jobs(keywords, location, max_results=50):
    jobs = []
    for start in range(0, max_results, 25):
        url = (
            "https://www.linkedin.com/jobs-guest/jobs/"
            "api/seeMoreJobPostings/search"
            f"?keywords={keywords}"
            f"&location={location}"
            f"&start={start}"
        )
        resp = requests.get(url, headers=HEADERS)
        if resp.status_code != 200:
            break

        soup = BeautifulSoup(resp.text, "lxml")
        cards = soup.select("div.base-card")

        if not cards:
            break

        for card in cards:
            title_el = card.select_one("h3.base-search-card__title")
            company_el = card.select_one("h4.base-search-card__subtitle")
            location_el = card.select_one("span.job-search-card__location")
            link_el = card.select_one("a.base-card__full-link")

            jobs.append({
                "title": title_el.text.strip()
                    if title_el else None,
                "company": company_el.text.strip()
                    if company_el else None,
                "location": location_el.text.strip()
                    if location_el else None,
                "url": link_el["href"].split("?")[0]
                    if link_el else None,
            })

        time.sleep(2)
    return jobs

results = search_jobs("data engineer", "San Francisco")
print(f"Found {len(results)} jobs")
Enter fullscreen mode Exit fullscreen mode

Getting Full Job Descriptions

The search endpoint gives you summaries. For full descriptions, scrape individual job pages:

def get_job_details(job_url):
    resp = requests.get(job_url, headers=HEADERS)
    soup = BeautifulSoup(resp.text, "lxml")

    desc_el = soup.select_one(
        "div.show-more-less-html__markup"
    )
    criteria = soup.select(
        "li.description__job-criteria-item"
    )

    details = {
        "description": desc_el.text.strip()
            if desc_el else None,
    }

    for item in criteria:
        label = item.select_one("h3")
        value = item.select_one("span")
        if label and value:
            key = label.text.strip().lower().replace(" ", "_")
            details[key] = value.text.strip()

    return details
Enter fullscreen mode Exit fullscreen mode

Approach 3: LinkedIn's Official API

LinkedIn does offer a Jobs API, but access is restricted:

  • Requires a LinkedIn developer account with approved use case
  • Limited to certain partner integrations
  • Rate limits are strict
  • Not available for general market research

For most use cases — especially research and analytics — the public jobs-guest endpoint is more practical.

Handling LinkedIn's Anti-Scraping Measures

LinkedIn is more aggressive about blocking scrapers than many sites:

  • IP blocking — rotate proxies or use residential IPs
  • Rate limiting — keep requests to 1 every 2-3 seconds minimum
  • HTML changes — LinkedIn updates its markup frequently; expect to update selectors
  • CAPTCHA — appears after sustained scraping from a single IP

Tips for reliability:

import random

def polite_request(url):
    delay = random.uniform(2.0, 4.0)
    time.sleep(delay)
    try:
        resp = requests.get(
            url, headers=HEADERS, timeout=10
        )
        resp.raise_for_status()
        return resp
    except requests.RequestException as e:
        print(f"Request failed: {e}")
        return None
Enter fullscreen mode Exit fullscreen mode

Comparison Table

Feature Apify Actors DIY Python LinkedIn API
Setup time Minutes Hours Days (approval)
Login required No (jobs-guest) No (jobs-guest) Yes
Proxy handling Built-in Manual N/A
Rate limits Managed Manual Strict
Full descriptions Usually yes Extra requests Varies
Cost Pay per usage Free + proxies Free (limited)

Legal Notes

The jobs-guest endpoint serves publicly accessible data to any visitor, logged in or not. The legal landscape:

  • The hiQ v. LinkedIn case established that scraping public data is not a CFAA violation
  • LinkedIn's Terms of Service prohibit scraping, but ToS violations are a civil matter, not criminal
  • Scraping for research, analytics, and building job tools is the lowest-risk use case
  • Avoid scraping personal profile data — stick to job postings

Getting Started

If you want results today: Search "linkedin jobs" on the Apify Store or check our upcoming LinkedIn Jobs scraper for a dedicated solution.

If you want full control: Start with the Python code above. The jobs-guest endpoint is stable and well-documented by the scraping community.

If you need scale: Use a managed platform with proxy rotation. Scraping 10,000+ job listings from a single IP will get you blocked within minutes.

LinkedIn job data is one of the most valuable datasets for understanding labor markets. With the right tools, you can access it programmatically and build powerful analytics on top of it.

Top comments (0)