DEV Community

Sebastian Wallinstabot project
Sebastian Wallinstabot project

Posted on

I stopped scraping Instagram — my AI agent pays per search instead (Python + x402)

#ai #python #webdev #machinelearning

The scraper treadmill

Every developer who has built an influencer tool or lead gen pipeline knows the cycle:

  1. Find an Instagram scraper library
  2. It works great for a week
  3. Instagram blocks your IP or rotates its internal API
  4. Find a proxy provider to stay ahead of bans
  5. Repeat every 3-6 months when the scraper breaks again

After going through this twice, I started looking for a stable alternative. I found one that uses a different model entirely: you pay per search instead of running a subscription.

How it works: x402 payments

Social Intel is an Instagram influencer search API that uses the x402 protocol — an HTTP extension where your client sends a USDC micropayment before getting data.

No API keys. No monthly plan. No rate limits per tier.

The flow in Python:

import httpx

# Step 1 — hit the endpoint, get 402 back
response = httpx.get(
    "https://socialintel.dev/v1/search",
    params={"query": "travel photographer", "country": "United States", "limit": 10}
)
# response.status_code == 402
# response.json() contains payment instructions

# Step 2 — pay $0.10 USDC on Base (handled by your x402 client)
# Step 3 — re-request with payment proof header
# Step 4 — get the data
Enter fullscreen mode Exit fullscreen mode

There are x402 client libraries for Python, JS, and Go. If you use AgentCash, the payment is completely transparent — you call fetch() and it handles 402 flows automatically.

What the data looks like

Free demo (try it without any wallet setup):

curl "https://socialintel.dev/v1/search?query=travel+photographer&limit=3&demo=true"
Enter fullscreen mode Exit fullscreen mode

Real output from that query:

{
  "demo": true,
  "results": [
    {
      "username": "bokehm0n",
      "full_name": "Johannes Hulsch | Travel Photographer",
      "followers": 548918,
      "is_verified": false,
      "public_email": "[available in paid response — ~50% of accounts]"
    },
    {
      "username": "tripwithmp",
      "full_name": "Mujeeb Padikka | Travel Photographer | Kerala | India",
      "followers": 452793,
      "is_verified": true,
      "public_email": "[available in paid response — ~50% of accounts]"
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode

The paid response adds: country, gender, category, bio, media count, and business email (~50% of profiles have one).

Filters that actually work 

results = client.search(
    query="yoga instructor",
    country="United Kingdom",   # or ISO code: "GB"
    gender="woman",             # "man" or "woman"
    min_followers=10000,
    max_followers=500000,
    is_verified=False,
    limit=50
)
Enter fullscreen mode Exit fullscreen mode

As an MCP tool (for AI agents)

If you are building with Claude or another MCP-compatible agent framework, there is a one-liner install:

uvx social-intel-mcp
Enter fullscreen mode Exit fullscreen mode

Your agent calls search_leads() and the x402 payment happens automatically — the MCP server manages the wallet. Your agent just gets results without any payment logic in your code.

The economics

Option Monthly cost for 100 searches
Modash / HypeAuditor $500+ (minimum plan)
Proxy + scraper $50-150
Social Intel (x402) $10

For hobbyist projects, internal tools, or proof-of-concepts: $10/month for on-demand data beats $500/month for a SaaS subscription you may not fully utilize.

Try the free demo: curl "https://socialintel.dev/v1/search?query=fitness+influencer&demo=true"

Full docs and quickstart: socialintel.dev/docs

Top comments (0)