DEV Community: Ocius Data

One API for Greenhouse, Lever, Ashby and Workday jobs: build a job feed that also knows when jobs close

Ocius Data — Mon, 02 Nov 2026 12:09:11 +0000

Disclosure: I build and maintain the Apify Actor used in this tutorial (ocius-data/ats-jobs-scraper). The ATS endpoints described below are public, and you can call them directly if you prefer.

If you build a job board, a recruiting tool, a sales-intelligence product ("which companies are hiring data engineers this month?") or a personal job alert, the cleanest source of jobs is not LinkedIn or Indeed. It is the applicant tracking system (ATS) each company uses to publish its openings. Most tech companies use one of four: Greenhouse, Lever, Ashby or Workday. Their public job boards are fresh (jobs appear the minute they are published) and free of reposts from aggregators.

The catch: each ATS has a different endpoint, a different schema and different quirks.

Greenhouse: boards-api.greenhouse.io/v1/boards/{token}/jobs. Simple JSON, salaries only on some boards.
Lever: api.lever.co/v0/postings/{company}, plus a separate EU host for EU tenants.
Ashby: api.ashbyhq.com/posting-api/job-board/{name}, with structured compensation.
Workday: a POST endpoint per tenant and site that stops at 2,000 results per search, so large boards are silently truncated unless you split the search.

This tutorial shows how to get all four in one normalized schema, and how to run it as a monitor that returns only new jobs and jobs that closed.

You need Python 3.11 or later, a free Apify account with its API token (Console, Settings, API and Integrations) and the official client: pip install "apify-client>=3" pandas.

Step 1: jobs from several companies in one call

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")

run = client.actor("ocius-data/ats-jobs-scraper").call(run_input={
    "companies": [
        "https://boards.greenhouse.io/stripe",
        "lever:spotify",
        "ashby:ramp",
        "https://nvidia.wd5.myworkdayjobs.com/NVIDIAExternalCareerSite",
    ],
    "keywords": ["engineer"],
    "maxJobsPerCompany": 50,
})

for job in client.dataset(run.default_dataset_id).iterate_items():
    print(job["ats"], job["company"], job["title"], job["location"])

Inputs can be board URLs, explicit forms such as greenhouse:stripe, bare names such as stripe (looked up on Greenhouse, Lever and Ashby), or a company's own careers page that links to one of these systems. Workday always needs the full career site URL, because Workday tenant names cannot be guessed.

Step 2: one schema for every ATS

Every row has the same fields, whatever the source:

jobKey: a stable unique key (ats:board:jobId) to deduplicate and update rows
title, department, team, location, countryCode
workplaceType (REMOTE, HYBRID, ONSITE) and isRemote
employmentType normalized to FULL_TIME, PART_TIME, CONTRACT, TEMPORARY or INTERNSHIP
salaryMin, salaryMax, salaryCurrency, salaryPeriod and salarySource (ats when it came from a structured field, description when it was parsed from the text)
descriptionText, descriptionHtml, jobUrl, applyUrl, postedAt

That makes questions across companies trivial:

import pandas as pd

df = pd.DataFrame(client.dataset(run.default_dataset_id).iterate_items())

# Remote share per company
print(df.groupby("company")["isRemote"].mean().round(2))

# Published salary ranges for engineering roles, annual USD only
pay = df[(df["salaryCurrency"] == "USD") & (df["salaryPeriod"] == "YEAR")]
print(pay[["company", "title", "salaryMin", "salaryMax"]].sort_values("salaryMax", ascending=False).head(10))

Step 3: a monitor that returns only what changed

For a job board or an alert, you do not want the full list every day. You want new jobs, and you want to remove jobs that were filled or taken down. Two options handle this:

run = client.actor("ocius-data/ats-jobs-scraper").call(run_input={
    "companies": ["https://boards.greenhouse.io/stripe", "lever:spotify", "ashby:ramp"],
    "keywords": ["engineer"],
    "onlyNewSinceLastRun": True,
    "reportClosedJobs": True,
    "monitoringStoreName": "engineering-monitor",
})

for row in client.dataset(run.default_dataset_id).iterate_items():
    if row["recordType"] == "closed":
        print("CLOSED", row["jobKey"], row["closedDetectedAt"])
    else:
        print("NEW", row["jobKey"], row["title"])

The first run returns every matching job and stores what was delivered.
Later runs return only jobs not delivered before.
With reportClosedJobs, a job missing from two complete listings in a row is reported once as recordType: "closed". The two-run confirmation prevents false closures when an ATS has a temporary error, and a partial or failed listing never produces closures.

Save the input as a task, schedule it every few hours, and connect a webhook or Google Sheets (use jobKey as the unique column). Your job board now adds new jobs and removes dead ones without manual cleanup.

Step 4: large Workday boards

Workday returns at most 2,000 results per search. The Actor detects when a board is larger and splits the search by job category and location until every job is read. In testing, one board returned 2,679 of 2,679 jobs.

Cost

At the time of writing: USD 0.003 per unique job on the Apify Free plan, less on paid plans, and USD 0.003 per closed job reported. Companies with no jobs, filtered-out jobs, duplicates and errors are free. One company with 200 open jobs costs about USD 0.60 the first time; a daily monitor afterwards pays only for new and closed jobs. Check the Actor page for current pricing.

Try it: ATS Jobs API on Apify. If you need another ATS (SmartRecruiters, Personio, Recruitee, Workable), open an issue on the Actor page: requests are counted and prioritized.

Runnable versions of every example in this article: apify-data-apis-examples on GitHub.

This article was drafted with AI assistance and reviewed, with every code sample tested against the live API.

How to monitor your competitors' Google Ads with the Ads Transparency Center (API, no login)

Ocius Data — Mon, 02 Nov 2026 11:59:18 +0000

Disclosure: I build and maintain the Apify Actor used in this tutorial (ocius-data/google-ads-transparency-reliable).

Since 2023, Google publishes every ad shown through Google Ads in the Ads Transparency Center: text, image and video ads, the countries where each one ran, and the first and last date it was shown. It is the most complete free source of competitor ad intelligence available, and it has one problem: there is no official API and no export button. You can browse one advertiser at a time, and that is all.

This tutorial shows how to turn it into a dataset you can query, and how to receive only the ads that started or stopped since the last check.

What you can get

For each ad:

advertiserId, advertiserName and creativeId
format (TEXT, IMAGE or VIDEO)
firstShown, lastShown, daysShown and an isActive flag
regions and per-region dates and, inside the EU, impression ranges and platforms
Ad text (headline, description, callToAction), imageUrl, videoUrl and landingUrl

With the optional full-creative mode, every variation of the ad: all headlines, all descriptions, every image and video.

You need Python 3.11 or later, a free Apify account with its API token (Console, Settings, API and Integrations) and the official client: pip install "apify-client>=3" pandas.

Step 1: all the ads pointing to a domain

The most useful query is a domain, not an advertiser name. A domain returns every ad that sends traffic to that website, from all the advertiser accounts behind it, including agencies and resellers.

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")

run = client.actor("ocius-data/google-ads-transparency-reliable").call(run_input={
    "searchQueries": ["nike.com"],
    "region": "US",
    "maxAdsPerAdvertiser": 100,
})

for ad in client.dataset(run.default_dataset_id).iterate_items():
    print(ad["format"], ad["daysShown"], ad.get("headline"), ad.get("landingUrl"))

maxAdsPerAdvertiser is enforced in the code, so you never pay for more ads than you asked for. Inputs with no ads, invalid domains and errors are reported in the run summary and are free.

Step 2: answer real questions with the data

Load the results into pandas and a few lines answer the questions a media buyer actually asks:

import pandas as pd

df = pd.DataFrame(client.dataset(run.default_dataset_id).iterate_items())

# Which formats does the competitor invest in?
print(df["format"].value_counts(normalize=True).round(2))

# Long-running ads are usually the winners: the advertiser keeps paying for them.
winners = df[df["isActive"]].sort_values("daysShown", ascending=False)
print(winners[["daysShown", "headline", "landingUrl"]].head(10))

# Which landing pages receive paid traffic?
print(df["landingUrl"].str.split("?").str[0].value_counts().head(10))

The "days shown" ranking is the most valuable view for creative research: an ad that has run for 400 days is one the advertiser keeps paying for, which is the closest public signal to "this ad converts".

Step 3: track changes instead of downloading everything again

Downloading every ad every day is wasteful. Monitoring mode keeps a snapshot per advertiser and returns only what changed:

run = client.actor("ocius-data/google-ads-transparency-reliable").call(run_input={
    "searchQueries": ["competitor-a.com", "competitor-b.com", "competitor-c.com"],
    "maxAdsPerAdvertiser": 200,
    "monitorChanges": True,
    "monitorStoreName": "my-competitors",
    "stoppedAfterDays": 3,
})

for row in client.dataset(run.default_dataset_id).iterate_items():
    print(row["changeType"], row["advertiserName"], row.get("headline"))

The first run stores a baseline and returns every current ad with changeType: "NEW".
Later runs return only NEW ads, ads that STOPPED (not shown for the number of days you set) and ads that RESUMED.

Save this input as a task in Apify Console and schedule it daily. Connect the run to a webhook, Slack, Google Sheets, Zapier, Make or n8n and you have a competitor ad alert system that costs cents per day.

Step 4: full creatives for a swipe file

When you need every variation of each ad (for example, to build a swipe file or to feed creatives to an AI model for analysis), turn on fullCreative:

run_input = {"searchQueries": ["nike.com"], "region": "US",
             "maxAdsPerAdvertiser": 50, "fullCreative": True}

Each row then includes allHeadlines, allDescriptions, allImageUrls and allVideoUrls.

Cost

At the time of writing: USD 0.0015 per ad saved (details included), USD 0.003 extra per ad with full creatives, and USD 0.002 per STOPPED or RESUMED row in monitoring mode. A weekly check of five competitors with 50 ads each costs about USD 0.38. Check the Actor page for current pricing.

Limits worth knowing

Impression ranges and platforms are published by Google only for ads shown in the EU.
Some text ads exist only as rendered images; their text is not available as structured fields, but imageUrl and adText are.
The Transparency Center shows ads, not spend or targeting.

Try it: Google Ads Transparency Center API on Apify. Feature requests are welcome in the Actor's Issues tab.

Runnable versions of every example in this article: apify-data-apis-examples on GitHub.

This article was drafted with AI assistance and reviewed, with every code sample tested against the live API.

Google Trends API in Python without pytrends 429 errors (and with more than 5 terms on one scale)

Ocius Data — Mon, 02 Nov 2026 11:58:23 +0000

Disclosure: I build and maintain the Apify Actor used in this tutorial (ocius-data/google-trends-reliable). The method for comparing more than five terms works with any tool; the code below uses ours.

Google has never shipped a public Google Trends API. Most Python projects use pytrends, an unofficial library that calls the same endpoints as the website. It works well in a notebook and then breaks in production for two reasons:

TooManyRequestsError (HTTP 429). Google rate-limits by IP address. A scheduled job on a cloud server hits the limit after a few requests, and the usual fix (time.sleep plus retries) makes every run slow and still unreliable.
The five-term limit. Google Trends compares at most five terms per request, and every request is scaled to its own 0-100 range. If you request terms in two batches, "60" in batch A and "60" in batch B do not mean the same thing.

This tutorial shows how to get interest over time, interest by region and related queries from Python with a single function call, and how to compare 10, 50 or 500 terms on one common scale.

What you need

Python 3.11 or later.
A free Apify account and its API token (Console, Settings, API and Integrations). The free plan includes monthly platform credit, enough to follow this tutorial.
The official client (version 3.x) and pandas: pip install "apify-client>=3" pandas. In version 3.x, call() returns a Run object, so the dataset ID is run.default_dataset_id.

Step 1: interest over time for one keyword

from apify_client import ApifyClient
import pandas as pd

client = ApifyClient("YOUR_APIFY_TOKEN")

run = client.actor("ocius-data/google-trends-reliable").call(run_input={
    "searchTerms": ["air conditioner"],
    "geo": "US",
    "timeRange": "today 12-m",
})

rows = list(client.dataset(run.default_dataset_id).iterate_items())
df = pd.DataFrame(rows)
print(df["type"].value_counts())

One run returns every data type in the same dataset, distinguished by the type field:

interestOverTime: one row per week (or day, or hour, depending on the range) with date, value and an isPartial flag for the current period.
interestByRegion: one row per state or country.
relatedQuery: top and rising related queries.

Retries, session rotation and the switch from datacenter to residential proxies happen inside the run, so your script never sees a 429. If a term returns no data, it is reported in the run summary with the status NO_DATA and is not charged.

Step 2: build a clean time series

iot = df[df["type"] == "interestOverTime"][["date", "value", "isPartial"]]
iot = iot[~iot["isPartial"]].set_index("date")
print(iot.tail())

Dropping the partial period avoids the classic mistake of reading the current, incomplete week as a sudden drop.

Step 3: compare more than five terms on one scale

This is where most projects get stuck. The standard workaround is an anchor term: a term that appears in every batch of five. You measure how the anchor scores in each batch and rescale the other four terms by the same factor. The Actor does this for you when compare is on:

terms = ["coffee", "tea", "beer", "wine", "water", "juice", "soda",
         "kombucha", "energy drink", "sparkling water"]

run = client.actor("ocius-data/google-trends-reliable").call(run_input={
    "searchTerms": terms,
    "compare": True,
    "anchorTerm": "coffee",
    "geo": "US",
    "timeRange": "today 12-m",
    "includeInterestByRegion": False,
    "includeRelatedQueries": False,
})

rows = client.dataset(run.default_dataset_id).iterate_items()
df = pd.DataFrame([r for r in rows if r["type"] == "interestOverTime"])

wide = df.pivot_table(index="date", columns="searchTerm", values="valueComparable")
print(wide.mean().sort_values(ascending=False).round(1))

valueComparable is a single 0-100 scale across all ten terms, where 100 is the highest interest of any term in the period. If the anchor term had very low interest in any batch, every row of the run is marked lowPrecision: true and the run log shows a warning; in that case, choose a more popular anchor instead of trusting a noisy number.

Choose a popular anchor. If your terms are niche ("heat pump water heater"), use a mid-volume term from the same topic rather than a giant term such as "weather", which would compress everything else toward zero.

Step 4: the searches trending right now

The same Actor reads the Google Trends "Trending now" page:

run = client.actor("ocius-data/google-trends-reliable").call(run_input={
    "mode": "trendingNow",
    "geo": "US",
    "trendingHours": "24",
    "maxTrendingSearches": 25,
})
for r in client.dataset(run.default_dataset_id).iterate_items():
    print(r["rank"], r["searchTerm"], r["searchVolumeLabel"], r["increasePercent"])

Step 5: run it on a schedule

In Apify Console, save your input as a task and add a schedule (for example, every Monday at 07:00). Each run stores its dataset, and you can read the newest one from Python, push it to Google Sheets, or receive a webhook when the run finishes.

Cost

The Actor charges per search term that returns data, with all data types included (USD 0.01 per term at the time of writing; check the Actor page for current pricing). A weekly check of ten keywords costs about USD 0.10. Terms without data and failed terms are free.

When pytrends is still the right choice

If you run a handful of queries by hand from your own computer, pytrends is free and fine. Move to a managed run when you need scheduled jobs on a server, more than five comparable terms, or results you can rely on without babysitting retries.

Try it: Google Trends API on Apify. Questions or feature requests are welcome in the Actor's Issues tab.

Runnable versions of every example in this article: apify-data-apis-examples on GitHub.

This article was drafted with AI assistance and reviewed, with every code sample tested against the live API.