DEV Community

Coco
Coco

Posted on • Edited on

How to scrape LinkedIn in 2026?

#python #webscraping #tutorial #api

Three LinkedIn scraping jobs in Python with no session cookie anywhere: profiles, company pages and job listings. Every request below was executed on 27 July 2026 against Chocodata and the numbers are that run.

HTTP 200 in 3.07s
fields: 12
             name: Reid Hoffman
         headline: Co-Founder, Executive Board Chair
  current_company: Manas AI
        followers: 2782185
Enter fullscreen mode Exit fullscreen mode

TL;DR

  • linkedin/profile takes a profile URL and returns 12 top-level fields in about 3 seconds, with no data envelope to unwrap.
  • linkedin/company returns 13 fields of firmographics, including employee_count at 16,850 for Stripe against a banded company_size of 5,001-10,000.
  • linkedin/jobsearch returned 10 live postings for backend engineer in the United Kingdom in 1.47s, each with a direct url.
  • specialties is a comma string on some company pages and None on others, so .split(",") raises AttributeError unless you guard it.

Why is it hard to scrape LinkedIn?

LinkedIn treats anonymous traffic as hostile: most of a profile sits behind an auth wall, class names are hashed and rotate so saved selectors rot within weeks, and rate limits attach to the account rather than the IP. The expensive one is HTTP 999, the non-standard code LinkedIn returns to suspected automation, because retry logic written for 429 does not recognise it and keeps hammering until the identity is burned. That mismatch costs more debugging time than the markup churn.

Prerequisites

  1. A free Chocodata API key, taken from the dashboard after sign-up. Free tier, no card.

Copying the free API key from the dashboard

  1. Python 3.9+ and requests. Tested here on Python 3.13.7 with requests 2.34.2, July 2026. Other languages hit the same endpoints, but every snippet below is Python. 
python -m venv .venv && . .venv/bin/activate
pip install requests
Enter fullscreen mode Exit fullscreen mode
  1. The LinkedIn URL you want to scrape. A profile URL copied from the address bar is enough to start. No LinkedIn account is involved at any point.

Fetch a profile from its URL

To scrape a LinkedIn profile, send the profile URL as url and read the fields straight off the response. There is no wrapper object.

1. Send the URL and assert on a field, not the status 

import time

import requests

BASE = "https://api.chocodata.com/api/v1"
API_KEY = "YOUR_API_KEY"
URL = "https://www.linkedin.com/in/reidhoffman"

r = requests.get(f"{BASE}/linkedin/profile",
                 params={"url": URL, "api_key": API_KEY}, timeout=90)
r.raise_for_status()
profile = r.json()

assert profile.get("name"), "empty payload behind a 200"
print("fields:", len(profile))
for k in ("id", "name", "headline", "location", "current_company", "followers", "connections"):
    print(f"{k:>16}: {profile[k]}")
Enter fullscreen mode Exit fullscreen mode 
fields: 12
              id: reidhoffman
            name: Reid Hoffman
        headline: Co-Founder, Executive Board Chair
        location: United States
 current_company: Manas AI
       followers: 2782185
     connections: 500+
Enter fullscreen mode Exit fullscreen mode

Terminal showing 12 fields and the parsed profile identity values

2. Iterate the experience and education lists 

print("experience:", len(profile["experience"]), "| education:", len(profile["education"]))

for e in profile["experience"][:6]:
    print("  ", e["company"])
for s in profile["education"][:4]:
    print("  ", s["school"])
Enter fullscreen mode Exit fullscreen mode 
experience: 13 | education: 6
   Manas AI
   Inflection AI
   Greylock
   Village Global
   Microsoft
   Aurora
   Università degli Studi di Perugia
   University of Oulu
   Babson College
   Oxford University
Enter fullscreen mode Exit fullscreen mode

Experience companies and education schools iterated from the lists

3. Flatten the nested lists into one CSV row 

import csv

row = {
    "id": profile["id"],
    "name": profile["name"],
    "headline": profile["headline"],
    "location": profile["location"],
    "current_company": profile["current_company"],
    "followers": profile["followers"],
    "companies": "|".join(e["company"] for e in profile["experience"]),
    "schools": "|".join(s["school"] for s in profile["education"]),
}

with open("profiles.csv", "w", newline="", encoding="utf-8") as fh:
    w = csv.DictWriter(fh, fieldnames=list(row))
    w.writeheader()
    w.writerow(row)

print(len(row["companies"].split("|")), "companies flattened")
Enter fullscreen mode Exit fullscreen mode 
13 companies flattened
Enter fullscreen mode Exit fullscreen mode

Pipe-joining keeps one profile on one row, which matters once you enrich thousands of them. Company names pulled out of experience are the handles the next section takes.

The profile flattened into a single CSV row

Fetch a company page by handle

To scrape a LinkedIn company page, pass the handle as company. The response carries the firmographics a profile does not.

1. Request the page and print the identity fields 

r = requests.get(f"{BASE}/linkedin/company",
                 params={"company": "stripe", "api_key": API_KEY}, timeout=90)
r.raise_for_status()
company = r.json()

print("fields:", len(company))
for k in ("id", "name", "industry", "founded", "headquarters"):
    print(f"{k:>14}: {company[k]}")
Enter fullscreen mode Exit fullscreen mode 
fields: 13
            id: stripe
          name: Stripe
      industry: Technology, Information and Internet
       founded: 2010
  headquarters: South San Francisco, California
Enter fullscreen mode Exit fullscreen mode

Company identity fields printed from the response

2. Read the size and reach signals 

print(f'employee_count : {company["employee_count"]:,}')
print(f'company_size   : {company["company_size"]}')
print(f'followers      : {company["followers"]:,}')
print(f'website        : {company["website"]}')
Enter fullscreen mode Exit fullscreen mode 
employee_count : 16,850
company_size   : 5,001-10,000 employees
followers      : 1,591,183
website        : https://stripe.com
Enter fullscreen mode Exit fullscreen mode

employee_count is the number of profiles listing that page as their employer, so it drifts a few people between calls and is not the same measure as the banded company_size.

Employee count, size band, followers and website

3. Batch several handles into one table 

def company_row(handle):
    r = requests.get(f"{BASE}/linkedin/company",
                     params={"company": handle, "api_key": API_KEY}, timeout=90)
    r.raise_for_status()
    return r.json()

for handle in ("stripe", "spotify", "microsoft"):
    d = company_row(handle)
    print(f'{d["name"]:<10}{d["employee_count"]:>9,}{d["followers"]:>12,}  {d["industry"][:34]}')
Enter fullscreen mode Exit fullscreen mode 
Stripe       16,850   1,591,183  Technology, Information and Intern
Spotify      19,179   4,628,644  Musicians
Microsoft   233,255  28,719,471  Software Development
Enter fullscreen mode Exit fullscreen mode

Note Spotify's industry reading as Musicians, which is the value on the page rather than a parsing artefact, so normalise industry yourself if you plan to group on it. Hiring volume is the other size signal, and that comes from the jobs endpoint.

Three company handles batched into one comparison table

Search jobs by keyword and location

To scrape LinkedIn jobs, pass keywords and location. The response is a ranked list of live postings, each with a direct URL.

1. Run the search and inspect the row shape 

r = requests.get(f"{BASE}/linkedin/jobsearch",
                 params={"keywords": "backend engineer",
                         "location": "United Kingdom",
                         "api_key": API_KEY}, timeout=90)
r.raise_for_status()
jobs = r.json()

print("results:", len(jobs["results"]))
print("keys:", [k for k in jobs["results"][0] if k != "company_logo"])
Enter fullscreen mode Exit fullscreen mode 
results: 10
keys: ['position', 'id', 'job_id', 'title', 'url', 'company', 'company_url', 'location', 'posted_date', 'posted_label', 'salary']
Enter fullscreen mode Exit fullscreen mode

Result count and the keys on a single job row

2. Print the listings as a fixed-width table 

for j in jobs["results"][:6]:
    print(f'{j["position"]:>2} {j["posted_date"]} {j["company"][:22]:<24}{j["title"][:34]}')
Enter fullscreen mode Exit fullscreen mode 
 1 2026-07-08 Happl                   Backend Engineer
 2 2026-07-19 Deliveroo               Software Engineer
 3 2026-06-17 Conduct                 Backend Engineer
 4 2026-07-19 MAI UK                  Backend Engineer
 5 2026-07-24 Midjourney              Backend Engineer, Core Systems
 6 2026-07-16 nPlan                   Backend Engineer
Enter fullscreen mode Exit fullscreen mode

Job listings printed as a fixed-width table

3. Filter to fresh postings by date 

CUTOFF = "2026-07-01"

fresh = [j for j in jobs["results"] if j["posted_date"] >= CUTOFF]
fresh.sort(key=lambda j: j["posted_date"], reverse=True)

print(f"{len(fresh)} of {len(jobs['results'])} posted since {CUTOFF}")
for j in fresh[:5]:
    print(f'  {j["posted_date"]}  {j["posted_label"]:<12}{j["company"][:20]}')
Enter fullscreen mode Exit fullscreen mode 
6 of 10 posted since 2026-07-01
  2026-07-24  2 days ago  Midjourney
  2026-07-21  6 days ago  Thought Machine
  2026-07-19  1 week ago  Deliveroo
  2026-07-19  1 week ago  MAI UK
  2026-07-16  1 week ago  nPlan
Enter fullscreen mode Exit fullscreen mode

posted_date is an ISO string, so string comparison sorts it correctly with no datetime parsing. Position 1 in the raw response was dated 2026-07-08 while position 5 was 2026-07-24, so rank is not recency.

Job rows filtered to postings after the cutoff date

The part that breaks

Optional nested fields, not the requests. specialties came back as a 23-item comma string for microsoft and as None for stripe, and founded did the opposite, None for Microsoft and 2010 for Stripe. Calling a string method on either one raises:

Traceback (most recent call last):
  File "scrape.py", line 57, in <module>
    tags = company["specialties"].split(",")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'split'
Enter fullscreen mode Exit fullscreen mode

Coalesce before you touch it, and the same guard covers every optional field on the page:

raw = company.get("specialties") or ""
tags = [t.strip() for t in raw.split(",") if t.strip()]
Enter fullscreen mode Exit fullscreen mode

Three more shapes worth knowing. Inside experience, only company was populated on both profiles tested, with title and date_range unset, so build on company names rather than job titles. connections came back 500+ for reidhoffman and None for williamhgates, so it is a display string and not a number. And salary was None on all 10 United Kingdom job rows, because most public postings do not publish a figure, which makes posted_date and company the only fields worth filtering on.

Full script 

"""LinkedIn profile, company and job scraper. Python 3.13.7, requests 2.34.2."""
import csv
import sys
import time
import requests

BASE = "https://api.chocodata.com/api/v1"
API_KEY = "YOUR_API_KEY"


def call(path, params, timeout=90, tries=3):
    """One GET, retried, so a batch run survives a dropped response."""
    params = dict(params, api_key=API_KEY)
    for attempt in range(tries):
        r = requests.get(f"{BASE}/{path}", params=params, timeout=timeout)
        if r.ok:
            return r.json()
        time.sleep(2)
    r.raise_for_status()


def profile(url):
    p = call("linkedin/profile", {"url": url})
    if not p.get("name"):
        raise RuntimeError(f"empty payload for {url}")
    return {
        "id": p["id"],
        "name": p["name"],
        "headline": p["headline"],
        "location": p["location"],
        "current_company": p["current_company"],
        "followers": p["followers"],
        "companies": "|".join(e["company"] for e in p.get("experience") or []),
        "schools": "|".join(s["school"] for s in p.get("education") or []),
    }


def company(handle):
    c = call("linkedin/company", {"company": handle})
    raw = c.get("specialties") or ""
    return {
        "id": c["id"],
        "name": c["name"],
        "industry": c["industry"],
        "founded": c.get("founded"),
        "headquarters": c["headquarters"],
        "employee_count": c["employee_count"],
        "company_size": c["company_size"],
        "followers": c["followers"],
        "specialties": [t.strip() for t in raw.split(",") if t.strip()],
    }


def jobs(keywords, location, since=None):
    rows = call("linkedin/jobsearch",
                {"keywords": keywords, "location": location})["results"]
    if since:
        rows = [j for j in rows if j["posted_date"] >= since]
    rows.sort(key=lambda j: j["posted_date"], reverse=True)
    return rows


def main():
    p = profile("https://www.linkedin.com/in/reidhoffman")
    print(f'{p["name"]} | {p["current_company"]} | {p["followers"]:,} followers')

    with open("profiles.csv", "w", newline="", encoding="utf-8") as fh:
        w = csv.DictWriter(fh, fieldnames=list(p))
        w.writeheader()
        w.writerow(p)

    for handle in ("stripe", "spotify", "microsoft"):
        c = company(handle)
        print(f'{c["name"]:<10}{c["employee_count"]:>9,}  {len(c["specialties"])} specialties')

    rows = jobs("backend engineer", "United Kingdom", since="2026-07-01")
    print(f"{len(rows)} fresh postings")
    for j in rows[:5]:
        print(f'  {j["posted_date"]}  {j["company"][:20]:<22}{j["url"]}')


if __name__ == "__main__":
    sys.exit(main())
Enter fullscreen mode Exit fullscreen mode

Summary

Three LinkedIn jobs run off one request shape and no login: a profile URL returns 12 top-level fields with experience and education as nested lists, a company handle returns 13 fields of firmographics, and a keyword plus a location returns ranked live postings with direct URLs. What the payload does not carry is anything gated behind a session, so this covers public research and lead enrichment rather than connection graphs. The habit that saves the most time is coalescing every optional field with or "" before you touch it, because specialties, founded, connections, salary and the nested job titles are all populated inconsistently from page to page.

FAQ

Do I need a LinkedIn login to run this?

No, every request here sends a URL or a handle plus an API key, with no session cookie and no account involved.

Why is specialties sometimes None?

Not every company page fills that section, so Stripe returned None while Microsoft returned a 23-item comma string, and calling .split(",") unguarded raises AttributeError.

Does the jobs endpoint return salary?

Salary came back None on all ten United Kingdom rows, because most public LinkedIn postings do not publish a figure.

Top comments (0)