DEV Community: member_924bafb1

Build a Hiring-Signal Feed from Greenhouse, Lever, Ashby and SmartRecruiters (with Python)

member_924bafb1 — Sun, 18 Oct 2026 04:39:24 +0000

"They just opened 6 SDR roles" is one of the strongest timing signals in outbound. If a company is hiring for the team that would use your product, you have a reason to reach out now. Most teams get that signal from LinkedIn scrapes or expensive data vendors. But most tech companies already publish their entire job board as clean JSON through their applicant tracking system (ATS). This tutorial shows how to turn that into a company-level hiring-signal feed you can refresh every week.

The data source: public ATS job-board APIs

Career pages are usually powered by one of a handful of ATSs, and each exposes a public, unauthenticated endpoint meant for embedding job lists:

ATS	Public endpoint pattern
Greenhouse	`https://boards-api.greenhouse.io/v1/boards/{slug}/jobs?content=true`
Lever	`https://api.lever.co/v0/postings/{slug}?mode=json` (EU: `api.eu.lever.co`)
Ashby	`https://api.ashbyhq.com/posting-api/job-board/{slug}?includeCompensation=true`
SmartRecruiters	`https://api.smartrecruiters.com/v1/companies/{id}/postings`

These are the employer's system of record. Roles show up here first and disappear when they're filled, so the data is fresher and cleaner than most aggregators.

A 20-line DIY version (Greenhouse only)

import httpx, collections

def greenhouse_signal(slug):
    jobs = httpx.get(f"https://boards-api.greenhouse.io/v1/boards/{slug}/jobs",
                     params={"content": "true"}, timeout=30).json()["jobs"]
    by_dept = collections.Counter(
        (j.get("departments") or [{"name": "Other"}])[0]["name"] for j in jobs)
    sales = [j for j in jobs if any(k in j["title"].lower()
             for k in ("sdr", "bdr", "account executive", "sales"))]
    return {"company": slug, "open_roles": len(jobs),
            "sales_roles": len(sales), "top_departments": by_dept.most_common(5)}

print(greenhouse_signal("stripe"))

That works, but a real feed also needs:

detection of which ATS a company uses,
Lever's EU cluster and SmartRecruiters' paging,
seniority classification,
salary parsing,
memory between runs, so you can say "+11 roles since last week".

That last part is what makes it a signal rather than just a list.

The hosted version: Hiring Signal Feed

Hiring Signal Feed is an Apify Actor that does all of the above. It returns one row per company, with:

open_roles, roles_by_department, roles_by_seniority, remote_roles
new_roles_7d and new_roles_30d
open_roles_change, roles_added_since_last_run and roles_closed_since_last_run, from a snapshot it keeps for you
salary_bands and the ats the company uses

Optionally, it also returns normalized job rows. Pricing is $0.01 per company row (or $0.0015 per job row), plus $0.02 per run.

pip install apify-client pandas

import os, pandas as pd
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
accounts = ["greenhouse:stripe", "greenhouse:discord", "ashby:ramp",
            "https://jobs.lever.co/spotify", "notion"]   # bare names are auto-detected

run = client.actor("eligible_fame/hiring-signal-feed").call(run_input={
    "companies": accounts,
    "outputMode": "companies",
    "titleKeywords": ["sales", "sdr", "bdr", "account executive"],
    "snapshotStoreName": "my-sales-hiring-watch",
})

df = pd.DataFrame(client.dataset(run["defaultDatasetId"]).iterate_items())
df = df[df.row_type == "company"]
print(df[["company", "ats", "open_roles", "new_roles_30d", "roles_added_since_last_run"]]
      .sort_values("new_roles_30d", ascending=False))

Because titleKeywords is set, every count is computed only on sales roles. So new_roles_30d = 9 means nine new sales jobs this month.

Score and route accounts

df["score"] = (df.new_roles_30d.fillna(0) * 2
               + df.roles_added_since_last_run.fillna(0) * 3
               + df.open_roles.fillna(0) * 0.2)
hot = df[df.score >= 10].sort_values("score", ascending=False)
hot.to_csv("hot_accounts.csv", index=False)

Get the actual postings for personalization

Run again with outputMode: "jobs" for just the hot accounts. Each job row includes title, department, location, posted_at, salary and the posting url. That's ideal for a first line like "Saw you're hiring a Founding AE in Austin…".

Make it weekly

Save the input as an Apify Task and attach a Schedule (for example, Mondays at 07:00).
Keep snapshotStoreName the same, so each run diffs against the last one.
Pipe results to your CRM:
- In Clay, use an HTTP API column that calls run-sync-get-dataset-items.
- In n8n, Make or Zapier, use the Apify app (Run Actor, then Get dataset items, then upsert into HubSpot or Sheets).

cURL for the HTTP column:

curl -X POST "https://api.apify.com/v2/acts/eligible_fame~hiring-signal-feed/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"companies":["{{domain_or_slug}}"],"outputMode":"companies"}'

Signals beyond sales hiring

The same feed answers other questions. A few common ones:

Selling data tools: filter for data engineer and analytics.
Selling security: watch for security and GRC roles.
Investors: track open_roles_change_pct across a portfolio for early momentum or slowdowns.
Recruiters: look for companies whose new_roles_7d jumps in your niche.

A sudden spike in manager and lead/principal seniority often means a new team is being built. That's usually a better trigger than one-off backfills.

Limitations to know

It only covers companies that use these four ATSs. That's most venture-backed tech, plus many enterprises on SmartRecruiters, but not every employer. If a company isn't found, you get a free error row.
posted_at semantics differ between ATSs. Some companies repost roles, so the snapshot diff (roles_added_since_last_run) is the most reliable "new" signal.
Salary only appears where the employer publishes it.

Wrap-up

Hiring data is the cheapest high-intent signal there is, because companies publish it themselves. Whether you write the 20-line script or use Hiring Signal Feed, the key is tracking change over time at the company level, not just collecting rows.

How to Get Google Trends Data in Python Without 429 Errors (a pytrends Alternative for 2026)

member_924bafb1 — Sun, 18 Oct 2026 04:38:51 +0000

If you've ever written pytrends.build_payload(...) and seen TooManyRequestsError: The request failed: Google returned a response with code 429, you're not alone. pytrends was archived, Google's official Trends API is still invite-only alpha, and Trends now rate-limits automated traffic aggressively. This tutorial shows a dependable way to pull interest over time, interest by region and related queries for hundreds of keywords from Python, and explains why the usual fixes stop working.

Why pytrends gets 429s

The Google Trends front end calls a handful of internal endpoints:

/trends/api/explore returns a list of "widgets", each with a short-lived token.
/trends/api/widgetdata/multiline returns the timeline.
/trends/api/widgetdata/comparedgeo returns regions.
/trends/api/widgetdata/relatedsearches returns related queries and topics.

Every response starts with an anti-JSON-hijacking prefix ()]}'), and every call needs a valid NID cookie. When you send many requests from one IP, Google returns 429 or redirects you to /sorry/. The common advice of time.sleep(60) plus retries=3 works for ten keywords. It doesn't work for a thousand, and it doesn't work from cloud IPs at all, because those are often blocked from the very first request.

What actually works is:

a fresh session (cookie plus new tokens) after each block,
rotating egress IPs,
per-widget retries, so one failed section doesn't throw away the whole query,
honest status reporting, so you know which rows are complete.

You can build that yourself, or use a hosted Actor that already does it.

Option A: a hosted API (about 5 minutes)

Reliable Google Trends API on Apify handles the warmup, proxy rotation, retries and residential fallback. It charges $0.002 per successful query, and failed queries are free. In our 34-query gate runs on Apify, it returned 34 of 34 results in every proxy mode we tested.

Install the client:

pip install apify-client pandas

Run a comparison:

import os
import pandas as pd
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])

run = client.actor("eligible_fame/reliable-google-trends").call(run_input={
    "searchTerms": [
        "chatgpt, claude, gemini",     # one query comparing 3 terms
        "air fryer",                   # a second query
    ],
    "geo": "US",
    "timeframe": "today 12-m",
    "interestOverTime": True,
    "interestByRegion": True,
    "relatedQueries": True,
})

rows = list(client.dataset(run["defaultDatasetId"]).iterate_items())
for r in rows:
    print(r["keywords"], r["status"], r["errors"])

Turn the timeline into a DataFrame

def timeline_df(row):
    recs = [{"date": p["time"], **p["values"]} for p in row["interest_over_time"]]
    return pd.DataFrame(recs).set_index("date")

df = timeline_df(rows[0])
print(df.tail())
df.plot(title="ChatGPT vs Claude vs Gemini, US, 12 months")

Rising queries (breakouts)

for kw, rq in rows[1]["related_queries"].items():
    for q in rq["rising"][:10]:
        print(kw, "->", q["query"], q["formatted_value"])

Bulk keyword lists

Each searchTerms entry is its own query, so you can pass 500 entries in one run. To set different options for each query, use customQueries:

run_input = {"customQueries": [
    {"keywords": "bitcoin, ethereum", "geo": "GB", "timeframe": "today 5-y", "property": "news"},
    {"keywords": "lofi music", "property": "youtube"},
]}

Every run also writes a RUN_SUMMARY record to the key-value store with the success rate and block counts:

summary = client.key_value_store(run["defaultKeyValueStoreId"]).get_record("RUN_SUMMARY")["value"]
print(summary)

Option B: roll your own (what you need to handle)

If you'd rather run it yourself, here's the minimum:

import httpx, json, time, random

def warm(client):
    # A headers-only request is enough to obtain an NID cookie
    with client.stream("GET", "https://trends.google.com/trends/explore?geo=US") as r:
        pass

def explore(client, kw, geo="US", tf="today 12-m"):
    req = {"comparisonItem": [{"keyword": kw, "geo": geo, "time": tf}],
           "category": 0, "property": ""}
    r = client.get("https://trends.google.com/trends/api/explore",
                   params={"hl": "en-US", "tz": 0, "req": json.dumps(req)})
    if r.status_code == 429:
        raise RuntimeError("blocked")
    return json.loads(r.text[r.text.index("{"):])["widgets"]

From here you still need to: call each widget endpoint with its token; rotate the session and the IP whenever you get a 429 or /sorry/; back off with jitter; and treat related topics as optional (Google often withholds them from sessions it labels USER_TYPE_SCRAPER). In practice the proxy bill and the maintenance cost far more than the code.

Gotchas that apply either way

Values are relative (0–100), not search volume. Only compare keywords that were in the same query.
More than 5 keywords? Use a shared anchor term in every batch (for example "weather") and rescale each batch against it.
Timeframe strings follow the Trends URL format: now 7-d, today 3-m, today 5-y, all, 2024-01-01 2024-06-30.
Last data point is often is_partial: true. Drop it before you fit a forecast.

Schedule it

In the Apify console, open the Actor, save your input as a Task, and add a Schedule (for example, daily at 06:00). Then use a webhook or the n8n/Make/Zapier Apify app to push new rows into Google Sheets or your warehouse.

Wrap-up

pytrends taught a generation of analysts to love Trends data, but the 429 wall is real. If you need a few queries a week, a careful DIY script with backoff can be enough. If you need hundreds of keywords on a schedule, without babysitting proxies, try Reliable Google Trends API. It's pay-per-result, and failed queries are free.