DEV Community

Aurelix Solutions
Aurelix Solutions

Posted on

Wellfound startup jobs with salary and equity as numbers, in Python

#python #jobs #startup #automation

Wellfound, the job board that used to be AngelList Talent, is where many early-stage startups post their roles, and most listings show pay up front: "$130k – $210k • 0.05% – 0.2%". That one line is the useful part for a job seeker comparing offers, a recruiter benchmarking a role, or an investor watching who is hiring. On the site it is text on a card, though, and you cannot sort a search by salary, filter by equity or ask what is new since yesterday.

This post pulls Wellfound jobs into Python with salary and equity as numbers, ranks a search by pay, and sets up a daily feed of new postings.

Disclosure: I'm Aurelix, and I built the Wellfound Jobs API & Scraper. It's a paid tool (prices below).

What each row holds

One row per job, 41 fields:

  • title, role, employment_type (FULL_TIME, CONTRACT...), years_experience_min and years_experience_max.
  • salary_min, salary_max, salary_currency, salary_interval, and salary_text as Wellfound shows it.
  • equity_min_pct, equity_max_pct, has_equity, and the original line in compensation_text.
  • The company: company, company_url, company_tagline, company_size (e.g. 11-50), company_stage (e.g. Early Stage), company_badges and actively_hiring.
  • locations, remote_locations, countries as ISO codes, remote and workplace_type (onsite, remote, onsite_or_remote).
  • posted_at (ISO 8601 UTC), job_url, description_text, and contact_emails: the addresses written in the description.

A search ranked by pay 

pip install apify-client
Enter fullscreen mode Exit fullscreen mode 
import os

from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("aurelix/wellfound-jobs-scraper").call(
    run_input={
        "roles": ["software-engineer", "data scientist"],
        "locations": ["new-york", "san francisco"],
        "jobTypes": ["FULL_TIME"],
        "salaryOnly": True,
        "maxJobs": 500,
    }
)
jobs = list(client.dataset(run["defaultDatasetId"]).iterate_items())

usd = [j for j in jobs if j["salary_currency"] == "USD" and j["salary_interval"] == "year" and j["salary_max"]]
for j in sorted(usd, key=lambda j: j["salary_max"], reverse=True)[:15]:
    equity = f'{j["equity_max_pct"]}%' if j["has_equity"] and j["equity_max_pct"] else "-"
    print(f'{j["salary_min"]:>8,}-{j["salary_max"]:<8,} eq {equity:>6}  {j["company_stage"] or "":14} '
          f'{j["company"][:24]:24} {j["title"][:40]}')
Enter fullscreen mode Exit fullscreen mode

roles and locations are Wellfound's own role and location pages: type words (product manager, San Francisco) or the slug from a wellfound.com/role/... link. Each role is combined with each location. startUrls takes role, location or remote links copied from the browser. titleKeywords then keeps only titles containing one of your words.

Pay by startup stage

Because salary and equity are numbers, benchmarking is a groupby:

from collections import defaultdict
from statistics import median

by_stage = defaultdict(list)
for j in usd:
    by_stage[j["company_stage"] or "unknown"].append((j["salary_min"] + j["salary_max"]) / 2)
for stage, mids in sorted(by_stage.items(), key=lambda x: -median(x[1])):
    print(f"{stage:16} {len(mids):4} jobs  median midpoint ${median(mids):,.0f}")
Enter fullscreen mode Exit fullscreen mode

The same rows split by company_size, workplace_type or years_experience_min answer what a role pays at a 11-50 person startup versus a 51-200 one, or remote versus onsite. "$130k – $210k • 0.05% – 0.2%" is parsed to 130000-210000 USD per year and 0.05-0.2% equity; currency comes from the symbol.

Remote jobs only 

run_input = {"roles": ["product-manager"], "remoteOnly": True, "maxJobs": 200}
Enter fullscreen mode Exit fullscreen mode

With roles and no locations, remoteOnly searches Wellfound's remote role pages and keeps jobs marked fully remote or onsite-or-remote.

New postings every morning 

run_input = {
    "roles": ["software-engineer"],
    "locations": ["new-york"],
    "incremental": True,
    "stateKey": "ny-engineers",
    "slackWebhookUrl": "https://hooks.slack.com/services/...",
}
Enter fullscreen mode Exit fullscreen mode

With incremental on, the first run returns every matching job; each later run with the same stateKey returns only jobs not returned before, remembered for 90 days per search. Save the input as a task in the Apify Console and schedule it daily: the Slack URL gets the count and the top 10 new jobs with links. webhookUrl posts the same digest as JSON to Zapier, Make or n8n. Wellfound does not sort its pages by date, so every page is read each day; a page with nothing new costs $0.0004, and a monitor of one role and city reads about 18 pages a day.

Why this tool

  • Salary and equity ranges as numbers, with currency and interval.
  • Startup stage, size, badges and tagline on every row.
  • Remote, job type, title and salary-published filters.
  • An only-new mode with a Slack or webhook digest.
  • What it doesn't cover: applying. That needs a Wellfound account; apply_url is the job's Wellfound page.

Limits

  • At most 3 jobs per startup per search. That is Wellfound's own limit on role and location pages; narrower searches (role plus city, or remote role pages) surface more of each startup's jobs.
  • No free-text search. Roles and locations are Wellfound's own pages; title keywords filter them.
  • At most 25 searches per run (roles times locations, plus links).
  • $ is read as USD, even for a startup posting in Canadian or Australian dollars.

Cost

$0.50 per 1,000 jobs ($0.40 on paid Apify plans), plus a $0.003 start fee per run and $0.0004 per results page that has no job to return. A role or location Wellfound does not have, and a page Wellfound refuses, are never charged. The current prices are on the Store page; Apify's free plan includes $5 of credit a month, about 9,900 jobs.

Links

Top comments (0)