Short Play Skits

Posted on Dec 15

I Tested Reddit's Free Traffic Thing for 3 Weeks (Here's What Actually Happened)

#webdev #programming #ai #beginners

So I needed to scrape Reddit posts.

Simple problem, right? Fetch some JSON, parse it, show it in a UI. Should be a weekend project.

Except Reddit's API is designed to punish exactly this use case.

The Problem with Web Apps

Here's the thing about Reddit's API - it has rate limits tied to OAuth clients, not users.

60 requests per minute per client.

If you're building a web app, all your users share the same client ID. Ten concurrent users? You're hitting rate limits constantly. And when you hit rate limits, Reddit doesn't throttle your app.

They ban the users' accounts.

Not ideal when you're trying to build something people actually use.

The Desktop Solution

I built Reddit Toolbox as an Electron app instead.

Each user runs their own instance. Each instance is its own OAuth client (or just scrapes public endpoints). Rate limits are per-user, not shared.

Plus local storage means I can cache data and filter client-side without hitting Reddit's servers repeatedly.

Stack:

Electron (cross-platform desktop)
Python backend (Reddit scraping)
SQLite (local data storage)
No central server

The lack of a server is key. Zero infrastructure costs. Zero shared rate limits. User owns their data.

Scraping Without OAuth

For public endpoints, I skip OAuth entirely.

import requests

url = f"https://www.reddit.com/r/{subreddit}/new.json?limit=100"
headers = {"User-Agent": "Mozilla/5.0"}
response = requests.get(url, headers=headers)
posts = response.json()['data']['children']

Public endpoints don't require authentication. Rate limits are IP-based and way more forgiving when it's one user on one IP.

Each user has their own IP. Problem solved.

Local-First Data Model

All scraped data goes into SQLite:

CREATE TABLE posts (
    id TEXT PRIMARY KEY,
    subreddit TEXT,
    title TEXT,
    num_comments INTEGER,
    score INTEGER,
    created_utc INTEGER
);

CREATE INDEX idx_comments ON posts(num_comments);

This lets me filter instantly without re-fetching from Reddit.

Want posts with less than 5 comments? SQL query takes milliseconds. No API calls.

SELECT * FROM posts 
WHERE num_comments < 5 
ORDER BY created_utc DESC;

The whole interface feels instant because nothing's waiting on network requests.

Why This Matters

I built this because I was trying to find Reddit threads to reply to for marketing.

The sweet spot is threads with 3-5 comments. Early enough that your reply won't get buried. Late enough that the thread has traction.

Manually browsing? Takes hours. Web scraper? Gets rate limited and bans your account.

Desktop app with local filtering? Problem solved.

The Distribution Trade-off

Desktop apps are harder to distribute than web apps.

Web app: Click URL, instant access.

Desktop app: Download installer, deal with OS warnings for unsigned builds, manual updates.

But when the API limits make a web app unusable? Desktop is the only option.

I'm distributing through a direct download on wappkit.com. No app store approval needed. Users get the binary, run it locally, done.

What I'd Do Differently

If I were rebuilding this:

1. Use Tauri instead of Electron

Electron bundles Chromium with every app. 200MB just for a simple UI. Tauri uses the OS's native webview. Way smaller binaries.

2. Add auto-updates

Right now updates are manual. Pain for users. Should've built in auto-update from day one.

3. Consider a hybrid model

Desktop app for scraping, optional web dashboard for viewing data. Best of both worlds.

The Bigger Point

Not everything needs to be a web app.

Sometimes the constraints of the platform you're integrating with (Reddit's API in this case) make desktop the better choice.

Local-first architectures have real advantages:

No server costs
Instant UI (no network latency)
User owns their data
No shared rate limits

Trade-off is distribution complexity. But for tools like this? Worth it.

Try It Yourself

Reddit Toolbox is free at wappkit.com if you want to test this workflow.

Code's not open source yet but planning to release the scraping engine once it's cleaner.

Curious what other people are building locally-first. Always interested in architecture discussions.

Building for hackers who need Reddit data: wappkit.com

Drop a comment if you've hit similar API limit problems. How'd you solve it?

DEV Community