Aurelix Solutions

Posted on Nov 5

Export new LinkedIn jobs to Google Sheets every day with Python

#python #automation #googlesheets #webscraping

If you run a niche job board, a hiring newsletter, or you are just job hunting in one country, you probably want the same thing: one spreadsheet that gains a row for every new posting that matches your search, every day, with no duplicates and no manual copy-paste.

This post walks through how to build that with Python. The general pattern works with any job source. I use a LinkedIn scraper for the data step, and the rest is plain Python and the Google Sheets API.

Disclosure: I'm Aurelix, and I built the LinkedIn Jobs Actor used below. It is a paid tool on the Apify Store; the pricing is in the Cost section, and the approach works with other sources too.

The general approach

A "daily export" sounds simple, but three details decide whether the sheet stays useful after a month:

A stable ID per job. Titles and companies repeat. You need an ID that is the same every time you see the same posting. On LinkedIn that is the number in /jobs/view/<id>/.
Incremental fetching. Each run should return only jobs you have not seen. You can do this on the source side (remember what was returned last time) or on your side (compare against IDs already in the sheet). Doing both is cheap insurance.
Append, never rewrite. People add notes and status columns to a shared sheet. If your script clears and rewrites the sheet, it destroys their work. Append new rows at the bottom and leave existing rows alone.

Then you schedule it: cron on a server, a GitHub Actions workflow, or a schedule on the platform that runs the scraper.

Step 1: Get the jobs

The LinkedIn Jobs Actor reads LinkedIn's public job pages (no login, no cookies) and returns one row per posting. The inputs that matter for a daily feed:

keywords: one search per entry, for example ["data engineer", "product manager"].
country: a two-letter code such as "CA" or "US". The Actor maps it to LinkedIn's exact country id and drops results whose location names another country.
datePosted: "any", "past24h", "pastWeek" or "pastMonth".
jobTypes: any of fullTime, partTime, contract, temporary, internship, volunteer, other.
maxResultsPerSearch: an exact cap per keyword (1 to 1,000, default 50).
incremental: when true, a run returns only jobs not returned by a previous run with the same searches.
stateKey: an optional name for that "seen before" memory, so two different feeds don't share it.

A full record looks like this (description fields shortened):

{
  "job_id": "4466544261",
  "title": "Data Engineer I",
  "company": "Docebo",
  "company_url": "https://www.linkedin.com/company/docebo",
  "location": "Toronto, Ontario, Canada",
  "countries": ["CA"],
  "posted_date": "2026-09-15",
  "posted_ago": "1 month ago",
  "job_url": "https://www.linkedin.com/jobs/view/4466544261/",
  "seniority": "Entry level",
  "employment_type": "Full-time",
  "industries": "Software Development",
  "applicants": "Over 200 applicants",
  "salary_text": "CA$68,600.00/yr - CA$91,500.00/yr",
  "description_text": "Artificial Intelligence. Actual Impact. ...",
  "description_html": "<strong>Artificial Intelligence. Actual Impact.<br/><br/></strong>...",
  "search_country": "CA",
  "source_url": "https://www.linkedin.com/jobs/view/4466544261/",
  "scraped_at": "2026-11-04T11:17:58Z"
}

job_id is the stable key we need.

Step 2: Prepare the Google Sheet

Create a Google Cloud project, enable the Google Sheets API, and create a service account. Download its JSON key.
Create a sheet and share it with the service account's email address (it looks like name@project.iam.gserviceaccount.com) with Editor access.
Copy the sheet ID from its URL: https://docs.google.com/spreadsheets/d/<SHEET_ID>/edit.

Install the two libraries:

pip install apify-client gspread

Step 3: The script

import os

import gspread
from apify_client import ApifyClient

APIFY_TOKEN = os.environ["APIFY_TOKEN"]
SHEET_ID = os.environ["SHEET_ID"]
GOOGLE_CREDS = os.environ.get("GOOGLE_CREDS", "service_account.json")

COLUMNS = [
    "job_id",
    "posted_date",
    "title",
    "company",
    "location",
    "seniority",
    "employment_type",
    "salary_text",
    "applicants",
    "job_url",
    "scraped_at",
]

RUN_INPUT = {
    "keywords": ["data engineer", "product manager"],
    "country": "CA",
    "datePosted": "pastWeek",
    "jobTypes": ["fullTime"],
    "maxResultsPerSearch": 100,
    "incremental": True,
    "stateKey": "sheet-ca-data-pm",
}


def fetch_new_jobs():
    client = ApifyClient(APIFY_TOKEN)
    run = client.actor("aurelix/linkedin-jobs").call(run_input=RUN_INPUT)
    return list(client.dataset(run["defaultDatasetId"]).iterate_items())


def to_row(job):
    return ["" if job.get(col) is None else str(job.get(col)) for col in COLUMNS]


def main():
    jobs = fetch_new_jobs()

    gc = gspread.service_account(filename=GOOGLE_CREDS)
    ws = gc.open_by_key(SHEET_ID).sheet1

    if not ws.row_values(1):
        ws.append_row(COLUMNS)

    known_ids = set(ws.col_values(1)[1:])
    rows = []
    for job in jobs:
        job_id = str(job.get("job_id"))
        if job_id in known_ids:
            continue
        known_ids.add(job_id)
        rows.append(to_row(job))

    rows.sort(key=lambda r: r[1], reverse=True)
    if rows:
        ws.append_rows(rows, value_input_option="RAW")

    print(f"{len(jobs)} jobs returned, {len(rows)} appended")


if __name__ == "__main__":
    main()

A few choices worth explaining:

datePosted: "pastWeek" plus incremental: True. The first run fills the sheet with the last week. After that, the incremental memory means each run only returns postings that weren't returned before, so a missed day (server down, laptop asleep) doesn't leave a gap, which a strict past24h window would.
A second dedupe against the sheet. If someone changes the stateKey, or you run the script from two places, the known_ids check still stops duplicate rows. Reading one column is cheap.
value_input_option="RAW". Values are stored as text, exactly as returned. Without it, Sheets may turn job IDs into numbers in scientific notation, and then your dedupe check stops matching.
No description column. Full descriptions run to thousands of characters and make a sheet hard to scan. Keep the job_url; if you want the text, add description_text to COLUMNS (a single Sheets cell holds up to 50,000 characters).

Step 4: Schedule it

On any Linux box, a cron entry is enough:

# every day at 07:00
0 7 * * * cd /opt/jobs-sheet && APIFY_TOKEN=... SHEET_ID=... /usr/bin/python3 export_jobs.py >> export.log 2>&1

If you don't want to keep a machine running, a scheduled GitHub Actions workflow works the same way: store APIFY_TOKEN, SHEET_ID and the service-account JSON as repository secrets and run the script on a schedule: trigger.

There is also a no-code route: schedule the Actor itself in Apify Console with incremental on, and connect its Google Sheets integration. That's fine for a single feed; the script above gives you control over columns, sorting and the extra dedupe.

Variations

Several countries. country takes one value per run. Call the Actor once per country with a different stateKey each time, and write each country to its own worksheet (gc.open_by_key(SHEET_ID).worksheet("US")).
Paste a search you already built. If you have a LinkedIn job search URL with filters set in the browser, put it in searchUrls instead of keywords. Its own filters are used as they are.
A cheaper "links only" feed. Set includeDetails to false to skip opening each job page. You get title, company, location, date and link only, and keyword matching is then checked against the title only.

Limits

Being honest about what this setup will and won't do:

LinkedIn shows at most about 1,000 jobs per search. For broad searches, split by keyword or location.
The Actor uses Apify's datacenter proxy only, and LinkedIn rate-limits datacenter addresses. On HTTP 429 it backs off and switches proxy sessions, but a search can still stop early; the jobs already found are kept and the run's status message names the search that stopped. For a daily feed of a few hundred jobs this matters less than for one big backfill.
posted_date is a date only. LinkedIn's public page shows no time.
salary_text is filled only where LinkedIn shows a salary on the public page, which is often not the case.
Workplace type (remote, hybrid, on-site) is sent to LinkedIn as a filter but can't be re-checked, because the public job page doesn't show it.
Keyword matching uses whole words, simple plurals and a few synonyms (for example RN for nurse). Very loose titles can be skipped.
Recruiter and poster names are never collected, and personal emails are removed from descriptions. Use the data for job search and hiring research, not for bulk outreach.

Why this tool over the alternatives

Compared with the leading alternatives on the Apify Store (the full table is on each Store page):

25% under the most-used LinkedIn jobs scraper on the free plan ($1.50 vs $2.00 per 1,000).
LinkedIn's exact country id with other-country results dropped, job type and level filters re-checked on each posting, and filtered-out jobs not charged. One rival is cheaper per job ($0.40).

Cost

At the time of writing, the README lists pay-per-result pricing: $1.50 per 1,000 jobs on Apify's free plan and $1.05 per 1,000 jobs on paid plans (Bronze, Silver, Gold), plus a start fee of $0.00005 per run. Jobs your filters drop and duplicates are not charged.

For the example above, the worst case is two keywords times 100 jobs = 200 jobs per run, or about $0.30 on the free plan. With incremental mode on, later runs usually return far fewer than the cap. Apify's free plan includes $5 of credit a month, which the README puts at about 3,300 jobs.

DEV Community