The Problem with "Public" Data
We all know that government data is technically "public," but that doesn't mean it's accessible. If you've ever tried to pull large datasets from government portals, you know the pain: antiquated UIs, weird CSV exports, and complex pagination.
I wanted to solve this for US Campaign Finance data. Journalists and researchers need to see who is funding whom, but they often spend more time cleaning data than analyzing it.
The Solution: Wrapping the OpenFEC API
We built the FEC Campaign Finance Scraper, an Apify Actor that wraps the official OpenFEC API to make this data consumable at scale.
Instead of writing custom scripts for every query, developers and researchers can now just input their parameters (committee IDs, date ranges, etc.) and get back clean, structured JSON.
Key Technical Challenges we solved:
- Pagination Hell: Automating the deep pagination required to fetch thousands of contribution records.
- Rate Limiting: Handling the FEC's API limits gracefully so the scraper doesn't get blocked mid-job.
- Data Normalization: structuring the output so it's ready for visualization libraries or SQL databases immediately.
The Unexpected Win!
We built this primarily to help the ecosystem, but this week we received some amazing validation. Apify selected our project as the winner of their "Social Good" Spotlight!
It was a great reminder that as developers, our side projects can have a real impact on transparency and democracy. It’s not always about building the next SaaS, sometimes it’s about making information free.
Try it out
If you are building in the Civic Tech or GovTech space, I’d love for you to stress-test it.
The Actor: https://apify.com/parseforge/fec-campaign-finance-contributions-scraper
The Goal: To help journalists track money in politics without needing a CS degree.
Question for the community: Have you worked with other government APIs? Which ones are the most painful to integrate? I’m looking for our next project!
Top comments (0)