I built LurkAPI: public social media data as JSON from one GET request

Mesmer Lord — Tue, 03 Nov 2026 23:12:31 +0000

I built LurkAPI, a social media data API. You send one GET request and get clean JSON from public pages: posts, profiles, comments, video transcripts, and the ads in Meta's, Google's, TikTok's and LinkedIn's ad libraries. It covers 16 platforms with 88 endpoints, and the same endpoints are an MCP server for AI apps.

One request

curl "https://api.lurkapi.com/v1/tiktok/profile?handle=nike" \
  -H "x-api-key: YOUR_API_KEY"

The answer, trimmed to a few fields:

{
  "success": true,
  "credits_remaining": 99985,
  "credits_charged": 1,
  "user": {
    "uniqueId": "nike",
    "nickname": "Nike",
    "signature": "Just Do It.",
    "verified": true
  },
  "stats": {
    "followerCount": 9253124,
    "heartCount": 49280939,
    "videoCount": 1089
  }
}

Every answer has the same envelope: success, credits_charged, credits_remaining, then the data. Errors carry a code and a link to the docs.

The same endpoints as an MCP server

Add https://api.lurkapi.com/mcp to Claude, Cursor, VS Code, Claude Code or Codex (ChatGPT takes https://api.lurkapi.com/mcp?compact, which has two tools instead of one per endpoint). You sign in through the browser, so there's no key to paste. Then ask in plain English: "What ads is Allbirds running on Facebook right now?"

What I cared about

Speed. Each request does the minimum: check the key, fetch, answer. It averages about a second.
Price. Most calls cost 1 credit. Packs start at $9 for 10,000 credits, credits never expire, and there's no subscription. Calls with bad parameters, rate-limited calls and 5xx errors are free.
Switching. 69 ScrapeCreators endpoints answer at the same path here, with the same x-api-key header, so moving is a change of base URL and key.
Docs. Every endpoint page has curl, Node.js, Python, Go, PHP and Ruby, a real captured response and a live playground. There's an OpenAPI 3.1 spec and llms.txt too.

It reads public data only: it never logs in to anyone's account and can't post or message anyone.

New accounts get 100 free credits with no card, and a balance under 25 tops back up to 25 every day.

Site: https://lurkapi.com
Docs: https://lurkapi.com/docs

I'd like to hear what's missing, or what breaks when you try it.

Why youtube-transcript-api gets blocked on servers, and what works

Mesmer Lord — Tue, 03 Nov 2026 22:56:20 +0000

Originally published at https://lurkapi.com/blog/youtube-transcript-api-blocked

When youtube-transcript-api works on your laptop and raises RequestBlocked on AWS, a VPS, Colab or a CI runner, your code is fine. YouTube is refusing your server’s IP: it answers requests from cloud and datacenter addresses with “Sign in to confirm you’re not a bot” instead of captions, and the library gives up in under a second.

From a fresh cloud server, 1 of 10 popular videos comes back; the other nine raise RequestBlocked. Two fixes still work: send the library through rotating residential proxies, or let a hosted transcript API make the request so your server never talks to YouTube. The third fix the error message offers, cookies, is switched off in the current release.

The block hides inside an HTTP 200

Each fetch is three requests. The library loads the video’s watch page to read an API key, posts to YouTube’s internal player endpoint as the Android app, then downloads the caption track the player lists. From a datacenter IP the first request goes through: about 1.3 MB of HTML, key included, no captcha. The second also returns 200, but its body says LOGIN_REQUIRED, with the bot check as the reason. The library raises RequestBlocked and never sends the third.

Both requests return 200; the second one’s body is the bot check.

So curling youtube.com from your server proves nothing: the page loads and every status code is 200. Without a proxy config the library doesn’t retry, and retrying yourself from the same IP gets the same answer.

IpBlocked is the other flavour: an HTTP 429, or a reCAPTCHA page where the watch page should be. It subclasses RequestBlocked, so one except RequestBlocked catches both. Versions before 1.0.0 reported the same block as TranscriptsDisabled (“Subtitles are disabled for this video”), so if an old pinned version swears a popular video has no subtitles, it’s probably being blocked too.

A green smoke test can hide a blocked server

On that blocked server, one video still worked: Rick Astley’s Never Gonna Give You Up, dQw4w9WgXcQ, the go-to test video. It came back all three times. Nine other popular videos raised RequestBlocked. A deploy check that fetches only that id goes green while the jobs behind it fail.

Check a server with the videos your job needs. If most of them raise RequestBlocked, it’s the IP.

Check your server · Python

from youtube_transcript_api import YouTubeTranscriptApi

api = YouTubeTranscriptApi()
ids = [  # use your own
    "dQw4w9WgXcQ",
    "kJQP7kiw5Fk",
    "arj7oStGLkU",
]
for vid in ids:
    try:
        t = api.fetch(vid)
        print(vid, "OK", len(t))
    except Exception as e:
        print(vid, type(e).__name__)

# From a cloud server:
# dQw4w9WgXcQ OK 61
# kJQP7kiw5Fk RequestBlocked
# arj7oStGLkU RequestBlocked

Cookies are off in 1.2.4, and proxies need upkeep

The error message offers cookies as its second fix, with a warning that YouTube will eventually ban the account you log in with. In 1.2.4, the current release, cookie auth is switched off (“Cookie auth has been temporarily disabled”), CLI flag included.

Its first fix is a proxy. The README recommends rotating residential proxies, because YouTube bans static ones after extended use. There’s a built-in config for one residential provider, and GenericProxyConfig for any other:

Any proxy · adapted from the README

from youtube_transcript_api import YouTubeTranscriptApi
from youtube_transcript_api.proxies import GenericProxyConfig

proxy = GenericProxyConfig(
    http_url="http://user:pass@my-custom-proxy.org:port",
    https_url="https://user:pass@my-custom-proxy.org:port",
)
ytt_api = YouTubeTranscriptApi(proxy_config=proxy)
ytt_api.fetch(video_id)

Know how retries behave before you buy traffic. The built-in provider config retries a blocked video up to 10 times; GenericProxyConfig tries once unless you subclass it and raise retries_when_blocked. Every attempt starts over at the watch page, so each refusal still costs a full page of traffic, and residential traffic is billed by the gigabyte.

If the videos are your own, there’s an official route. The YouTube Data API’s captions.download needs OAuth and permission to edit the video, and each call costs 200 of your 10,000 daily quota units, so 50 downloads a day. It can’t read anyone else’s videos.

Or take YouTube off your server: one GET

With a hosted API, your server makes one HTTPS call and the YouTube side happens elsewhere, so it stops mattering where your code runs. LurkAPI’s transcript endpoint fetches the captions for you, and a cached repeat comes back in 20 ms.

Same ten videos: 1 from the library, 10 from LurkAPI.

Video transcript: What's said in a video or Short: its captions as timed segments and as one block of text.

GET https://api.lurkapi.com/v1/youtube/video/transcript · 1 credit per call · Read the docs

Get a transcript · 1 credit

curl "https://api.lurkapi.com/v1/youtube/video/transcript?url=https://www.youtube.com/watch?v=arj7oStGLkU" \
  -H "x-api-key: $LURK_KEY"

The JSON, trimmed to 2 segments

Response

{
  "success": true,
  "credits_charged": 1,
  "videoId": "arj7oStGLkU",
  "url": "https://www.youtube.com/watch?v=arj7oStGLkU",
  "language": "English",
  "languageCode": "en",
  "isGenerated": false,
  "transcript": [
    { "text": "So in college,", "startMs": 12645, "endMs": 14015, "startTimeText": "0:12" },
    { "text": "I was a government major,", "startMs": 15349, "endMs": 16913, "startTimeText": "0:15" }
  ],
  "transcript_only_text": "So in college, I was a government major, …",
  "availableLanguages": [
    { "code": "ar", "name": "Arabic", "isGenerated": false },
    { "code": "hy", "name": "Armenian", "isGenerated": false },
    { "code": "bg", "name": "Bulgarian", "isGenerated": false }
  ]
}

Each call is 1 credit: $0.45 to $0.90 per 1,000 transcripts depending on the pack, and a new account starts with 100 free credits.

Switching from the library takes a few lines. Segments come back as text, startMs and endMs; to_snippet converts one to the library’s text, start and duration:

Python · 1 credit

import requests

API = ("https://api.lurkapi.com"
       "/v1/youtube/video/transcript")
KEY = "lk_live_..."

def transcript(video_id):
    r = requests.get(
        API,
        params={"url": video_id},
        headers={"x-api-key": KEY},
        timeout=30,
    )
    r.raise_for_status()
    return r.json()["transcript"]

def to_snippet(seg):
    s = seg["startMs"]
    e = seg["endMs"]
    return {
        "text": seg["text"],
        "start": s / 1000,
        "duration": (e - s) / 1000,
    }

segs = transcript("arj7oStGLkU")
print(len(segs), to_snippet(segs[0]))
# 315 {'text': 'So in college,',
#  'start': 12.645, 'duration': 1.37}

One default differs. The library asks for English only and raises NoTranscriptFound otherwise; the endpoint returns English when the video has it and falls back to the video’s main language when it doesn’t. Pass language=es (or pt-BR, and so on) for a specific track, and check languageCode if you need English. A video with no captions returns 404 not_found.

For a whole channel, list its uploads with channel videos and make one transcript call per video. For one transcript without code, the free YouTube transcript tool runs the same endpoint in your browser, or try it here:

Run GET /v1/youtube/video/transcript in the live playground

Questions

What does RequestBlocked mean in youtube-transcript-api?

YouTube answered the library’s player request with “Sign in to confirm you’re not a bot” instead of captions, almost always because the request came from a cloud or datacenter IP. IpBlocked, a subclass, means an HTTP 429 or a reCAPTCHA page; except RequestBlocked catches both.

Why does youtube-transcript-api work locally but not on a server?

Same code, different IP. YouTube blocks most IPs that belong to cloud providers such as AWS, Google Cloud and Azure, and a home IP can be blocked too after too many requests.

Do cookies work with youtube-transcript-api?

Not in 1.2.4. Cookie auth is switched off in the code, CLI flag included, even though the RequestBlocked message still suggests it.

How do I use a proxy with youtube-transcript-api?

Pass a proxy_config to YouTubeTranscriptApi: the built-in config for one residential provider, or GenericProxyConfig(http_url=..., https_url=...) for any other. The README recommends rotating residential proxies, since static ones get banned after extended use.

Is there an official YouTube transcript API?

Only for videos you can edit. The YouTube Data API’s captions.download needs OAuth and edit permission, and costs 200 of the default 10,000 daily quota units per call. For anyone else’s videos, LurkAPI’s transcript endpoint returns the captions for 1 credit a call.

DEV Community: LurkAPI