DEV Community

earnnova-dev
earnnova-dev

Posted on Originally published at github.com

I built a zero-dependency CLI to snapshot the remote job market into CSV/JSON/SQLite

I had a recurring need: a reproducible, offline snapshot of the remote-job market I could open in a spreadsheet, load into a database, or feed into a script — without scraping five job boards myself, juggling five different site formats, or maintaining five scraping pipelines that break every time a board changes their markup.

So I built remote-jobs-export: a small, stdlib-only (zero third-party dependencies) CLI that pulls normalized remote-job data from the free Remote Jobs API and writes it to CSV / JSON / SQLite — or prints a one-line summary. No API key needed for the free tier.

The problem I kept hitting

When I want to answer "what does the remote market look like right now?" I need a stable, queryable dataset. The pain was never the analysis — it was the ingest:

  • Each board (Remotive, RemoteOK, Jobicy, We Work Remotely, Hacker News) has its own shape, its own pagination, its own way of (not) exposing salary.
  • A scraper per board means five maintenance surfaces, each one a ticking time bomb.
  • I wanted the data local — in a file I could GROUP BY, join, diff against last week, or hand to a colleague — not "go to the website."

The Remote Jobs API already does the normalization (one clean schema, five sources, salaries parsed to a consistent floor/top in USD). What was missing for me was a local export layer on top of it. That's the gap remote-jobs-export fills.

What it does

pip install remote-jobs-export

# A 100-job snapshot into a CSV I can open in Excel / Google Sheets
remote-jobs-export --limit 100 -o jobs.csv

# The same data in SQLite for SQL analysis
remote-jobs-export --limit 200 -o jobs.db

# Or JSON to feed into my own pipeline
remote-jobs-export --limit 200 -o jobs.json

# Filter by source, skills, or minimum salary
remote-jobs-export --source remotive -o remotive.csv
remote-jobs-export --skills python,devops -o python_devops.csv
remote-jobs-export --min-salary 90000 -o senior.csv

# Just a quick summary, no file
remote-jobs-export --limit 300 --summary
Enter fullscreen mode Exit fullscreen mode

Because it's stdlib-only, there's no dependency hell and no vendored HTTP client that needs a pip install fix when a project upgrades its Python version. requests-style conveniences are all done with the standard library.

A real snapshot (captured live today)

remote-jobs-export --limit 120 --summary returned:

total: 120
by_source:  jobicy 99, wwr 21
by_category: (none) 99, Customer Support 7, All Other Remote 4,
             Sales and Marketing 4, Management and Finance 3,
             Full-Stack Programming 2, DevOps and Sysadmin 1, ...
Enter fullscreen mode Exit fullscreen mode

Two things are honest about that output, and I think they matter:

  1. category is frequently empty upstream — it's reported as (none), not silently dropped. A buyer of a "clean jobs dataset" should know the source data has gaps, not see them papered over.
  2. The numbers are the feed as-is. I don't invent a category to make the distribution look tidier. If the boards under-report category, the export shows that.

Why export-first (and not just "hit the API")

The API is live and queryable — and that's fine for a one-off curl. But the moment I want a snapshot in time — "the market on 2026-10-05" vs "the market on 2026-09-05" — I need the data materialized locally. A CSV/SQLite file is diffable, versionable, and shareable in a way a live endpoint isn't. That's the whole point of the export tool: it turns a live feed into an offline, queryable artifact.

What I'm proud of

  • Zero dependencies. It runs anywhere Python 3 does, no pip install surprises.
  • Honest by default. Empty upstream fields are shown as-is, salaries are parsed to a documented floor, and the summary reflects the live feed exactly.
  • Small and readable. The whole thing is one small package with a clear CLI surface.

Try it

pip install remote-jobs-export
remote-jobs-export --limit 300 --summary
Enter fullscreen mode Exit fullscreen mode

The tool is on PyPI and on GitHub. If you're pulling job-market data into a notebook, a dashboard, or a weekly report, I'd love to hear what you're doing with it — and any bugs you find.

Top comments (0)