I bootstrapped an API business on a single RTX 3060, from literally zero budget

JoaoPauloNA — Sun, 09 Aug 2026 19:47:22 +0000

Two weeks ago I had zero dollars, one gaming PC with an RTX 3060 Ti, and a stubborn idea: what if the cost of running a small, useful LLM-backed API could be close to zero, if you just... hosted the model yourself?

Here's what I shipped, and the specific things that broke along the way — because the failures were more instructive than the plan.

The idea

A small API that turns plain English into working code artifacts:

/v1/regex — "validate a Brazilian CEP" → a working regex, an explanation, and match/no-match examples
/v1/sql — "list the 10 customers who bought the most last month" → a SQL query
/v1/commit-message — a diff description → a conventional commit message
/v1/json-schema — "an e-commerce product with name, price, category" → a JSON Schema

Nothing revolutionary. The point wasn't the idea — it was proving the economics work when the model is yours.

The stack

Ollama running qwen2.5-coder:7b locally on a Windows box with an RTX 3060 Ti (8GB VRAM — plenty for a 7B coder model at Q4 quantization)
FastAPI wrapping it, with a JSON-only system prompt and a tolerant parser (models don't always respect "respond with JSON only," so I regex out the first valid {...} block if json.loads fails on the raw text)
Cloudflare Tunnel (the free "quick tunnel," not a named/paid one) to expose it publicly without port forwarding or a static IP
RapidAPI as the billing/marketplace layer — free tier + paid tier, they handle subscriptions so I don't have to build that

Total infrastructure cost: $0. The GPU was already sitting there.

What actually broke (the useful part)

Windows process persistence is not what you think. Anything you launch directly in an SSH session — even with Start-Process, even backgrounded — dies the moment the SSH session closes, because Windows ties it to a Job Object scoped to that session. The fix that actually works: wrap the command in a .bat, register it as a Scheduled Task (schtasks /create ... /sc onlogon), and trigger it once immediately with schtasks /run. That survives disconnects.

"Free static domain" isn't always what it says. I tried ngrok's free static domain (a real, permanent feature — not a myth) to get a stable URL instead of Cloudflare's rotating one. It works great in a browser. It's useless for an API: ngrok's free tier shows a mandatory interstitial warning page to any request that doesn't send a specific ngrok-skip-browser-warning header — and a marketplace proxy calling your API on a customer's behalf will never send that header. Every single API call returns an HTML warning page instead of JSON. I found this by testing the raw endpoint externally with no special headers before trusting the "solved" checkbox — which is the actual lesson: test the exact path a real client will take, not the happy path you control.

LLMs don't reliably return valid JSON, even when told to. The fix wasn't a smarter prompt — it was a tolerant extractor: try json.loads on the raw text, then try pulling a fenced


 block, then fall back to a brace-matching regex. Three tries, cheap, and it turned "the model sometimes wraps JSON in markdown" from a 502 error into a non-issue.

**A brand-new marketplace listing needs its own free tier.** I gated the public demo (`/demo/*`) with a simple in-memory per-IP daily counter (5 free tries/day) instead of requiring signup at all for the landing page. Letting people try before they subscribe converts better than a wall.

## The economics, honestly

RapidAPI takes 25% of marketplace revenue, plus a small payout processing fee. That's real, and it means the math on a $9.99/month plan is closer to $7 net than $10. Worth knowing before you price.

## What's next

The backend is done. The actual bottleneck now is distribution — a working API with zero users doesn't pay rent. If you're building something adjacent (dev tools, LLM-backed APIs, or you're just curious about the self-hosted-model economics), I'd genuinely like to hear what you're seeing.

**API listing:** [https://rapidapi.com/JoaoPauloNA/api/plain-english-to-code-api](https://rapidapi.com/JoaoPauloNA/api/plain-english-to-code-api) (free tier, no credit card to try the demo)

---

*Building this in public as I go — happy to answer questions about the self-hosted LLM economics, the FastAPI/Ollama integration, or the Windows automation quirks above.*

DEV Community: JoaoPauloNA

I bootstrapped an API business on a single RTX 3060, from literally zero budget

The idea

The stack

What actually broke (the useful part)