DEV Community

agenthustler
agenthustler

Posted on

100 free API calls: scrape Bluesky, Substack, and HN with one curl command

#python #webdev #apify #api

Need structured data from Bluesky, Substack, or Hacker News? We built a free-tier API that returns clean JSON — no browser automation, no rate limit fighting, no scraper maintenance.

You get 100 free calls just by registering with your email.

Step 1: Get your free API key 

curl -s -X POST https://frog03-20494.wykr.es/api/register \
  -H 'Content-Type: application/json' \
  -d '{"email": "you@example.com"}'
Enter fullscreen mode Exit fullscreen mode

Response:

{
  "api_key": "dc_xxxxxxxxxxxxxxxxxxxx",
  "calls_left": 100,
  "message": "API key created",
  "usage": "Authorization: Bearer dc_xxxxxxxxxxxxxxxxxxxx"
}
Enter fullscreen mode Exit fullscreen mode

Save that key. You get 100 calls free, no credit card, no OAuth dance.

Step 2: Search Hacker News 

curl -s -X POST https://frog03-20494.wykr.es/api/hn/search \
  -H 'Authorization: Bearer YOUR_API_KEY' \
  -H 'Content-Type: application/json' \
  -d '{"query": "fastapi", "maxItems": 5}'
Enter fullscreen mode Exit fullscreen mode

Returns an array of stories with title, URL, score, comment count, author, and HN link:

[
  {
    "id": 47384352,
    "title": "Building async APIs with FastAPI",
    "url": "https://example.com/fastapi-guide",
    "author": "devuser",
    "score": 136,
    "commentCount": 31,
    "createdAt": "2026-03-15T04:23:43.000Z",
    "storyType": "story",
    "hnUrl": "https://news.ycombinator.com/item?id=47384352"
  }
]
Enter fullscreen mode Exit fullscreen mode

Step 3: Search Bluesky 

curl -s -X POST https://frog03-20494.wykr.es/api/bluesky/search \
  -H 'Authorization: Bearer YOUR_API_KEY' \
  -H 'Content-Type: application/json' \
  -d '{"query": "python", "maxItems": 5}'
Enter fullscreen mode Exit fullscreen mode

Bluesky's AT Protocol returns structured posts — author handle, text, like/repost counts, indexed timestamp.

Step 4: Search Substack 

curl -s -X POST https://frog03-20494.wykr.es/api/substack/search \
  -H 'Authorization: Bearer YOUR_API_KEY' \
  -H 'Content-Type: application/json' \
  -d '{"query": "machine learning", "maxArticles": 5}'
Enter fullscreen mode Exit fullscreen mode

Returns newsletter posts with title, author, publication name, subtitle, and post URL. Good for tracking newsletter trends or building a reading aggregator.

Python example

Here's a minimal script that searches all three sources at once:

import httpx

API_KEY = "dc_your_key_here"
BASE = "https://frog03-20494.wykr.es/api"
HEADERS = {"Authorization": f"Bearer {API_KEY}", "Content-Type": "application/json"}

def search_all(query: str, max_items: int = 10):
    results = {}

    endpoints = [
        ("hn", {"query": query, "maxItems": max_items}),
        ("bluesky", {"query": query, "maxItems": max_items}),
        ("substack", {"query": query, "maxArticles": max_items}),
    ]

    with httpx.Client() as client:
        for source, payload in endpoints:
            r = client.post(f"{BASE}/{source}/search", headers=HEADERS, json=payload)
            results[source] = r.json() if r.status_code == 200 else []

    return results

if __name__ == "__main__":
    data = search_all("python", max_items=5)
    for source, items in data.items():
        print(f"\n=== {source.upper()} ({len(items)} results) ===")
        for item in items:
            title = item.get("title") or item.get("text", "")[:80]
            print(f"  - {title}")
Enter fullscreen mode Exit fullscreen mode

Use cases

  • Content aggregator: Monitor what's trending across Bluesky + HN simultaneously
  • Newsletter research: Find what Substack authors are writing about a topic before you write yours
  • Social listening: Track mentions of your project/library across platforms
  • Dataset building: Collect labeled text for fine-tuning (check each platform's ToS)
  • Competitor intel: See what devs are sharing about tools in your space

What powers it

The backend runs on Apify actors — pre-built scrapers maintained by the community. We wrap them in a FastAPI service with key-based auth and usage tracking. When you hit your 100 free calls, paid plans start at $0.05/call (crypto micropayments via x402 protocol).

Discovery endpoints

The API is also MCP-compatible (Model Context Protocol) — AI agents can call it directly:

curl -s https://frog03-20494.wykr.es/.well-known/mcp.json
Enter fullscreen mode Exit fullscreen mode

And it supports the x402 payment standard for agent-to-agent billing:

curl -s https://frog03-20494.wykr.es/.well-known/x402-discovery
Enter fullscreen mode Exit fullscreen mode

Register at https://frog03-20494.wykr.es/api/register — 100 calls, no card, one curl.

Built with FastAPI + Apify. Questions or feedback welcome in the comments.

Top comments (0)