DEV Community

agenthustler
agenthustler

Posted on

How to Scrape LinkedIn Job Listings in 2026 (Python + Public API, No Login Required)

#python #api #webdev #tutorial

LinkedIn is one of the largest job boards in the world, but it doesn't offer a free public API for job listings. The good news? You don't need one. LinkedIn exposes a public guest endpoint that serves job data without authentication.

In this guide, I'll show you how to scrape LinkedIn job listings in 2026 using Python — legally, efficiently, and without logging in.

How LinkedIn's Public Jobs Endpoint Works

LinkedIn serves job listings to non-logged-in visitors through a guest-facing API. When you visit a LinkedIn job search page without being signed in, your browser hits endpoints under linkedin.com/jobs-guest/. These return HTML that can be parsed for structured job data.

The two key endpoints:

  • Job search: https://www.linkedin.com/jobs-guest/jobs/api/seeMoreJobPostings/search?keywords={query}&location={location}&start={offset}
  • Job details: https://www.linkedin.com/jobs-guest/jobs/api/jobPosting/{job_id}

No API key. No OAuth. No login. These are public pages LinkedIn serves to search engines and anonymous visitors.

Is This Legal?

Scraping publicly accessible data is generally legal, especially after the hiQ Labs v. LinkedIn ruling where the court affirmed that scraping public data does not violate the Computer Fraud and Abuse Act. That said:

  • Only scrape public endpoints (no login required)
  • Respect robots.txt and rate limits
  • Don't scrape personal profile data — stick to job listings
  • Don't hammer their servers — add delays between requests

This guide only uses public, unauthenticated endpoints.

Scraping LinkedIn Job Listings with Python

Step 1: Search for Jobs 

import requests
from bs4 import BeautifulSoup
import time

def search_linkedin_jobs(keywords, location, num_jobs=25):
    jobs = []
    headers = {
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
                      "AppleWebKit/537.36 (KHTML, like Gecko) "
                      "Chrome/120.0.0.0 Safari/537.36"
    }

    for start in range(0, num_jobs, 25):
        url = (
            "https://www.linkedin.com/jobs-guest/jobs/api/"
            "seeMoreJobPostings/search"
            f"?keywords={keywords}"
            f"&location={location}"
            f"&start={start}"
        )

        response = requests.get(url, headers=headers)
        if response.status_code != 200:
            print(f"Got status {response.status_code}, stopping.")
            break

        soup = BeautifulSoup(response.text, "html.parser")
        job_cards = soup.find_all("div", class_="base-card")

        for card in job_cards:
            title_el = card.find("h3", class_="base-search-card__title")
            company_el = card.find("h4", class_="base-search-card__subtitle")
            location_el = card.find("span", class_="job-search-card__location")
            link_el = card.find("a", class_="base-card__full-link")

            jobs.append({
                "title": title_el.text.strip() if title_el else None,
                "company": company_el.text.strip() if company_el else None,
                "location": location_el.text.strip() if location_el else None,
                "url": link_el["href"].split("?")[0] if link_el else None,
            })

        time.sleep(2)  # Be respectful

    return jobs

# Example usage
results = search_linkedin_jobs("python developer", "United States", num_jobs=50)
for job in results[:5]:
    print(f"{job['title']} at {job['company']} - {job['location']}")
Enter fullscreen mode Exit fullscreen mode

Step 2: Get Job Details

Each job listing has a numeric ID in its URL. Use it to fetch the full description:

def get_job_details(job_id):
    url = f"https://www.linkedin.com/jobs-guest/jobs/api/jobPosting/{job_id}"
    headers = {
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
                      "AppleWebKit/537.36 (KHTML, like Gecko) "
                      "Chrome/120.0.0.0 Safari/537.36"
    }

    response = requests.get(url, headers=headers)
    if response.status_code != 200:
        return None

    soup = BeautifulSoup(response.text, "html.parser")

    description_el = soup.find("div", class_="show-more-less-html__markup")
    criteria = soup.find_all("li", class_="description__job-criteria-item")

    details = {
        "description": description_el.text.strip() if description_el else None,
        "criteria": {}
    }

    for item in criteria:
        label = item.find("h3")
        value = item.find("span")
        if label and value:
            details["criteria"][label.text.strip()] = value.text.strip()

    return details

# Extract job ID from URL and fetch details
job_url = "https://www.linkedin.com/jobs/view/3812345678"
job_id = job_url.split("/")[-1]
details = get_job_details(job_id)
if details:
    print(details["criteria"])
    print(details["description"][:500])
Enter fullscreen mode Exit fullscreen mode

Step 3: Extract Job IDs from Search Results

Connecting the two steps together:

def extract_job_id(url):
    if not url:
        return None
    parts = url.rstrip("/").split("/")
    return parts[-1] if parts[-1].isdigit() else None

# Full pipeline
results = search_linkedin_jobs("data engineer", "Remote", num_jobs=25)
for job in results[:3]:
    job_id = extract_job_id(job["url"])
    if job_id:
        details = get_job_details(job_id)
        print(f"\n{'='*60}")
        print(f"{job['title']} at {job['company']}")
        if details and details["criteria"]:
            for k, v in details["criteria"].items():
                print(f"  {k}: {v}")
        time.sleep(2)
Enter fullscreen mode Exit fullscreen mode

Handling Rate Limits and Blocks

LinkedIn will start returning 429 errors if you scrape too fast. A few practical tips:

  1. Add delays: 2-3 seconds between requests minimum
  2. Rotate User-Agents: Use a pool of realistic browser UA strings
  3. Use proxy rotation: Essential for any serious volume

Proxy Solutions for LinkedIn Scraping

For anything beyond light testing, you will want rotating proxies. I have had good results with ScraperAPI — it handles proxy rotation, retries, and CAPTCHAs automatically. You just prefix your target URL:

# Using ScraperAPI for proxy rotation
def search_with_proxy(keywords, location):
    target_url = (
        "https://www.linkedin.com/jobs-guest/jobs/api/"
        f"seeMoreJobPostings/search?keywords={keywords}"
        f"&location={location}&start=0"
    )

    api_url = (
        f"http://api.scraperapi.com"
        f"?api_key=YOUR_SCRAPERAPI_KEY"
        f"&url={target_url}"
    )

    response = requests.get(api_url)
    return response.text
Enter fullscreen mode Exit fullscreen mode

ScraperAPI offers a free tier with 5,000 requests, which is enough to test your pipeline. For production scraping, their plans handle the IP rotation and retry logic so you don't have to.

Pre-Built Solution: LinkedIn Jobs Scraper on Apify

If you don't want to build and maintain your own scraper, I have been working on a ready-to-use LinkedIn Jobs Scraper on Apify. It wraps the same public endpoints with built-in proxy rotation, structured JSON output, and scheduling — so you can set it up once and get fresh job data delivered daily.

It is currently being finalized and will be publicly available on the Apify store soon.

Storing Results

For anything beyond quick scripts, save to a structured format:

import csv

def save_to_csv(jobs, filename="linkedin_jobs.csv"):
    if not jobs:
        return
    keys = jobs[0].keys()
    with open(filename, "w", newline="", encoding="utf-8") as f:
        writer = csv.DictWriter(f, fieldnames=keys)
        writer.writeheader()
        writer.writerows(jobs)
    print(f"Saved {len(jobs)} jobs to {filename}")

results = search_linkedin_jobs("machine learning", "New York", num_jobs=50)
save_to_csv(results)
Enter fullscreen mode Exit fullscreen mode

Key Takeaways

  • LinkedIn's jobs-guest endpoints are public and don't require authentication
  • You can search jobs and fetch full descriptions with basic Python (requests + BeautifulSoup)
  • Respect rate limits — add delays, rotate user agents, and consider a proxy service like ScraperAPI for volume
  • For a managed solution, check out the LinkedIn Jobs Scraper on Apify
  • Always scrape responsibly — public data only, reasonable request rates, no personal data

The full code from this tutorial is straightforward to extend. Add filters for date posted, experience level, or job type by appending query parameters to the search URL (e.g., &f_TPR=r86400 for jobs posted in the last 24 hours, &f_E=2 for entry-level).

Happy scraping.

Top comments (0)