DEV Community

Big Man
Big Man

Posted on

How to get job listings from Google Jobs with an API (Python, Node or no code)

#webscraping #python #javascript #api

Google Jobs collects listings from LinkedIn, Indeed, company career pages and many smaller job boards, and shows them in one panel on the search page. That makes it a useful single source for job data, but Google has no public API for it.

This tutorial uses Google Jobs Scraper (US & UK), an Apify actor I built. For each search it returns the title, company, location, salary, job type, full description and the real apply links behind each listing. It also has an only-new mode for scheduled runs. The full examples are on GitHub: auto-actor/google-jobs-scraper.

What people use it for

  • A daily feed of new postings for a role and city with onlyNew: true, so you only see jobs you haven't seen.
  • Salary and job-type snapshots across cities.
  • Lists of employers hiring for a skill, with direct apply links.

Input 

{"queries": ["data analyst new york"], "country": "us", "maxItems": 10}
Enter fullscreen mode Exit fullscreen mode
Field What it does
queries Job searches, e.g. software engineer london.
country us or uk.
maxItems Stop after this many jobs in total.
onlyNew Skip jobs already returned by earlier runs (remembered in the google-jobs-seen store of your account). Ideal for schedules.
debug Save raw HTML for debugging.

Python 

"""Run Google Jobs Scraper (US & UK) on Apify and print the results.

    pip install apify-client
    export APIFY_TOKEN=...   # Apify Console -> Settings -> Integrations
    python example.py
"""
import os

from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])

run = client.actor("daedal_quintuplet/google-jobs-scraper").call(
    run_input={
        "queries": [
            "data analyst new york"
        ],
        "country": "us",
        "maxItems": 10
    }
)

jobs = client.dataset(run["defaultDatasetId"]).iterate_items()
for job in jobs:
    print(job["title"], "|", job["company"], "|", job["location"], "|", job.get("salary") or "no salary listed")
Enter fullscreen mode Exit fullscreen mode

Node.js 

// Run Google Jobs Scraper (US & UK) on Apify and print the results.
//
//   npm install apify-client
//   APIFY_TOKEN=... node example.js     (token: Apify Console -> Settings -> Integrations)

const { ApifyClient } = require('apify-client');

async function main() {
    const client = new ApifyClient({ token: process.env.APIFY_TOKEN });

    const run = await client.actor('daedal_quintuplet/google-jobs-scraper').call({
        "queries": [
            "data analyst new york"
        ],
        "country": "us",
        "maxItems": 10
    });

    const { items } = await client.dataset(run.defaultDatasetId).listItems();
    for (const job of items) {
        console.log(job.title, '|', job.company, '|', job.location, '|', job.salary || 'no salary listed');
    }
}

main().catch((err) => {
    console.error(err);
    process.exit(1);
});
Enter fullscreen mode Exit fullscreen mode

No code: one HTTP request

Works from n8n, Make, Zapier or curl:

curl -X POST "https://api.apify.com/v2/acts/daedal_quintuplet~google-jobs-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"queries": ["data analyst new york"], "country": "us", "maxItems": 10}'
Enter fullscreen mode Exit fullscreen mode

What you get

One row per job:

Field Meaning
id Google's job id
title Job title
company Employer
location Location text
via The job board Google sourced it from
postedAt Posting age as Google shows it
salary Salary text when shown
jobType Full-time, Part-time, Contractor, Internship
description Full job description
applyLinks Every apply option Google lists, with its URL
googleJobsUrl Link to the listing on Google Jobs
query The query that found it
country us or uk
scrapedAt When it was scraped

Limits

  • Google returns 10 jobs per query; use several specific queries rather than one broad one.
  • US and UK only. Google does not show a jobs panel for some countries, Australia included.

Price

$0.0025 per job ($2.50 per 1,000), paid per result, with no monthly fee. Apify's free plan includes $5 of usage a month, which is enough to try it.

Links: Store page · GitHub examples

Top comments (0)