DEV Community: Andrew

Automate your job search: a daily LinkedIn jobs pipeline in Python

Andrew — Tue, 11 Aug 2026 20:09:47 +0000

Job hunting has a timing problem that nobody warns you about.

A posting goes up. For the first few hours it has a handful of applicants. By day three it has two hundred, a recruiter has stopped reading carefully, and your carefully tailored application is row 187 in an ATS.

You cannot control how good the other 186 candidates are. You can control whether you were row 8 instead. That is an automation problem, and it takes about forty lines of Python.

The two filters that actually matter

LinkedIn's job search has a lot of knobs. For this purpose, two of them do most of the work:

Posted in the last 24 hours. Anything older is already contested.
Under 10 applicants. LinkedIn exposes this, and it is the single best proxy for "you will actually be read."

Everything else — seniority, remote, job type — narrows the funnel to roles you would actually take. But the two above are what turn a job feed into an edge.

Checking that by hand every morning is exactly the kind of thing you stop doing on day four. So let's not do it by hand.

One call for the data

I am using an Apify Actor that queries LinkedIn's job search and returns structured results. The endpoint below starts a run, waits, and hands back the rows — no polling loop, no login, no cookie to extract from your browser.

curl -X POST \
  "https://api.apify.com/v2/acts/data_pool~linkedin-jobs-scraper/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "keywords": ["backend engineer", "platform engineer"],
    "maxItems": 100,
    "location": "Berlin, Germany",
    "workplaceType": ["remote", "hybrid"],
    "datePosted": "24h",
    "sortBy": "date",
    "under10Applicants": true
  }'

Each keyword runs as its own search, and results are de-duplicated across them, so overlapping search terms do not produce duplicate rows.

What comes back

{
  "jobId": "4012345678",
  "jobUrl": "https://www.linkedin.com/jobs/view/4012345678/",
  "title": "Senior Backend Engineer",
  "location": "Berlin, Germany",
  "workplaceType": "hybrid",
  "listedAtIso": "2026-08-11T09:12:00Z",
  "easyApply": true,
  "fewApplicants": true,
  "isPromoted": false,
  "company": {
    "name": "Acme GmbH",
    "profileUrl": "https://www.linkedin.com/company/acme-gmbh"
  },
  "insights": ["Actively hiring"],
  "matchedKeyword": "backend engineer"
}

Three fields deserve attention:

jobId is stable across runs. That is your deduplication key — without it, a daily job will keep showing you Monday's postings all week.
workplaceType comes back as a raw enum (on_site, remote, hybrid), not a pretty label. Map it before it reaches a human.
isPromoted tells you the company paid to boost the listing. Not automatically bad, but a promoted post that is three weeks old is a different signal from a fresh organic one.

The daily script

This is the whole thing: fetch, drop anything seen before, print a digest.

import json
import os
import pathlib
import requests

ACTOR = "data_pool~linkedin-jobs-scraper"
URL = f"https://api.apify.com/v2/acts/{ACTOR}/run-sync-get-dataset-items"
SEEN = pathlib.Path("seen_jobs.json")

WORKPLACE = {"on_site": "On-site", "remote": "Remote", "hybrid": "Hybrid"}

SEARCH = {
    "keywords": ["backend engineer", "platform engineer"],
    "maxItems": 100,
    "location": "Berlin, Germany",
    "workplaceType": ["remote", "hybrid"],
    "datePosted": "24h",
    "sortBy": "date",
    "under10Applicants": True,
}


def fetch_jobs():
    resp = requests.post(
        URL,
        headers={"Authorization": f"Bearer {os.environ['APIFY_TOKEN']}"},
        json=SEARCH,
        timeout=300,
    )
    resp.raise_for_status()
    return resp.json()


def main():
    seen = set(json.loads(SEEN.read_text())) if SEEN.exists() else set()

    fresh = []
    for job in fetch_jobs():
        job_id = job.get("jobId")
        if not job_id or job_id in seen:
            continue
        seen.add(job_id)
        fresh.append(job)

    for job in fresh:
        company = (job.get("company") or {}).get("name", "unknown")
        flags = []
        if job.get("easyApply"):
            flags.append("Easy Apply")
        if job.get("fewApplicants"):
            flags.append("<10 applicants")
        suffix = f"  [{', '.join(flags)}]" if flags else ""
        print(f"{job['title']} — {company}")
        print(f"  {job.get('location', '')} · {WORKPLACE.get(job.get('workplaceType'), '')}{suffix}")
        print(f"  {job.get('jobUrl', '')}\n")

    SEEN.write_text(json.dumps(sorted(seen)))
    print(f"{len(fresh)} new posting(s).")


if __name__ == "__main__":
    main()

Run it, and you get only what appeared since yesterday.

One gotcha: this endpoint returns HTTP 408 if the run takes more than 5 minutes. For a daily search that is not a concern, but if you ever crank maxItems into the thousands, switch to the async pattern (POST /runs, then poll).

Actually running it every day

The script is deliberately boring so you can schedule it however you like:

cron, if you have a machine that is always on: 0 8 * * * cd ~/jobsearch && APIFY_TOKEN=... python daily.py >> log.txt
GitHub Actions on a schedule: trigger, with the token as a repository secret — free, and no machine to maintain.
n8n / Make / Zapier, if you would rather the output land in Google Sheets or a Slack DM than a terminal. The same call works as a single HTTP Request node.

The digest is more useful pushed somewhere you already look. Slack DM, a Telegram bot, or an email to yourself all beat a log file you forget to open.

Filters worth knowing about

Field	Why you'd use it
`under10Applicants`	The highest-signal filter here. Apply while the pile is small.
`datePosted: "24h"`	Keeps a daily run cheap and the results genuinely new.
`easyApply`	One-click applications, if you're playing a volume strategy.
`workplaceType`	`remote` is often geo-restricted anyway — pair it with `location`.
`seniority`	`entry_level` through `executive`; filters out the mismatches.
`company`	Watch one employer's hiring. Useful when you're targeting a shortlist.

What it costs

$0.50 per 1,000 job postings returned, with no per-run fee and no subscription.

A realistic daily search pulling 100 postings costs about $1.50 a month — comfortably inside Apify's free monthly credit, so a personal job search runs at no cost. The pricing only starts to matter if you are feeding a job board rather than a job hunt.

The honest caveats

LinkedIn caps what any search returns. Realistically a few hundred results per query, not every posting in existence. Several narrow searches beat one enormous one.
Results are not deterministic. LinkedIn's ranking shifts between runs, so treat a daily digest as "what surfaced today", not a complete index.
under10Applicants reflects LinkedIn's own counter, which updates on its own schedule. Treat it as a strong hint, not a guarantee.
The applicant count is not the whole story. Being early gets you read. It does not get you hired — that part is still on you and your CV.

Wrapping up

The interesting part of this was not the scraping. It was noticing that "posted in the last 24 hours" plus "under 10 applicants" is a genuinely different product from "job search", and that the only thing standing between those two things was a scheduled script and a stable ID to deduplicate on.

If you are hunting right now: automate the finding, and spend the time you save on the applications themselves.

Disclosure: I built the Actor used in this post. Costs quoted are its public list price in August 2026, and the caveats above are the same ones I would give you in person.

Export everyone who engaged with a LinkedIn post — no login, no cookies

Andrew — Tue, 11 Aug 2026 17:19:22 +0000

If you post on LinkedIn, the people who react and comment are the warmest leads you will ever get. They saw your thing, they cared enough to click, and they raised their hand in public.

Getting that list out of LinkedIn is annoying in three specific ways. This post is about making it one API call.

Why the usual approaches hurt

1. Most tools want your session cookie. You open DevTools, copy your li_at value, and paste it into someone else's server. Now your personal account is the one making automated requests, and if the tool gets rate-limited or flagged, it is your account that eats the restriction. That is a bad trade for a lead list.

2. Reactions and comments are usually two separate products. The two biggest tools in this niche sell "post comments" and "post reactions" as different scrapers. Someone who liked and commented shows up in both exports, so you run two jobs, download two files, and write dedup logic before you have anything usable.

3. The output is nested. You want a spreadsheet. You get JSON with a person object inside every row.

One call instead

I will use an Apify Actor that does reactions and comments in a single run and merges duplicate people. The endpoint below starts the run, waits, and returns the rows directly — no polling loop.

curl -X POST \
  "https://api.apify.com/v2/acts/data_pool~linkedin-post-engagement-scraper/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "postUrls": ["https://www.linkedin.com/feed/update/urn:li:activity:7491138591868469248/"],
    "engagementTypes": ["reactions", "comments"],
    "maxItemsPerPost": 100,
    "dedupePeople": true
  }'

postUrls accepts normal post links, lnkd.in short links, or raw urn:li:activity:... IDs, so you can throw whatever format you already have at it.

What comes back

One object per person:

{
  "postUrl": "https://www.linkedin.com/feed/update/urn:li:activity:7491138591868469248/",
  "postAuthorName": "Jane Doe",
  "engagementTypes": ["reaction", "comment"],
  "reactionValue": "PRAISE",
  "commentText": "This matches what we're seeing too.",
  "commentedAt": "2026-07-01T12:00:00Z",
  "person": {
    "name": "John Smith",
    "headline": "VP Engineering @ Acme",
    "profileUrl": "https://www.linkedin.com/in/john-smith",
    "type": "individual"
  }
}

Two things worth knowing about this shape, because they will bite you otherwise:

engagementTypes is an array. Someone who both reacted and commented has ["reaction", "comment"] in a single row rather than two rows. That is the dedup doing its job.
person.type is "individual" or "organization" — company pages react to posts too. If you are building an outreach list you almost certainly want to filter the organizations out.

Flattening it into a lead list

import os
import requests

ACTOR = "data_pool~linkedin-post-engagement-scraper"
URL = f"https://api.apify.com/v2/acts/{ACTOR}/run-sync-get-dataset-items"

def engagers(post_url, limit=100):
    resp = requests.post(
        URL,
        headers={"Authorization": f"Bearer {os.environ['APIFY_TOKEN']}"},
        json={
            "postUrls": [post_url],
            "engagementTypes": ["reactions", "comments"],
            "maxItemsPerPost": limit,
            "dedupePeople": True,
        },
        timeout=300,
    )
    resp.raise_for_status()

    for row in resp.json():
        person = row.get("person") or {}
        if person.get("type") != "individual":
            continue  # skip company pages
        yield {
            "name": person.get("name", ""),
            "headline": person.get("headline", ""),
            "profile_url": person.get("profileUrl", ""),
            "engagement": " + ".join(row.get("engagementTypes") or []),
            "comment": row.get("commentText", ""),
        }

for lead in engagers("https://www.linkedin.com/feed/update/urn:li:activity:7491138591868469248/"):
    print(lead)

That is a CSV away from being usable.

One gotcha: this endpoint returns HTTP 408 if the run takes longer than 5 minutes. Keep maxItemsPerPost in the low hundreds, or switch to the async pattern (POST /runs, then poll) for genuinely large jobs.

Doing it without code

If you would rather wire it into a workflow, the same call works as a single HTTP Request node in n8n, which is handy when the destination is Google Sheets or Slack rather than your own script. The only setup is a Header Auth credential holding Bearer <token>.

What it costs

Per 1,000 people returned, at list prices as of August 2026:

Tool	Reactions	Comments	Both from one post
This Actor	$1.50 / 1k	$1.50 / 1k	one run
Alternative A (~6,000 users)	$2.00 / 1k	$2.00 / 1k	two separate Actors
Alternative B (~4,800 users)	—	$5.00 / 1k	two separate Actors

The "both from one post" column is the part that actually matters. If you want reactors and commenters, the alternatives mean two runs and your own dedup pass, so the real gap is wider than the per-1000 numbers suggest.

Apify's free tier includes monthly credit, so a few thousand engagers costs nothing to try.

The honest caveats

Coverage is whatever LinkedIn surfaces for that post at that moment. Posts with tens of thousands of reactions come back partially. Nobody in this category gets 100%, whatever the marketing says.
Reactions have no timestamp. LinkedIn timestamps comments but not reactions, so commentedAt is empty for reaction-only rows. Do not build "engaged in the last 24h" logic on it.
This is personal data. Names, headlines, and profile URLs of identifiable people are personal data under GDPR and similar laws. Publicly visible is not the same as unrestricted — you still need a lawful basis to process it, and cold outreach in the EU has real requirements. Scraping it is the easy part; using it responsibly is the part worth thinking about.

Wrapping up

The pattern that made this simple was picking a tool that treats "everyone who engaged" as one dataset rather than two products, and using a sync endpoint so there is no polling loop. Both decisions removed more code than any clever scraping trick would have.

Disclosure: I built the Actor used in this post. The comparison figures are the list prices published on the Apify Store in August 2026; I have left the alternatives unnamed because the point is the structure of the pricing, not a swipe at anyone — the two tools referenced are easy to find and worth comparing for yourself. The caveats section is the same thing I would tell you if you asked me whether to buy it.