How I Turned My Own Project Into an Apify Actor

#api #saas #startup

I started building https://instadl.ink — a fast, no-storage media extractor for Instagram & TikTok. Paste a public URL → get direct CDN links. No permanent video storage, just JSON. When I wanted to monetize the API on Apify, every All-in-One downloader there was a yt-dlp wrapper. So I expanded instadl.ink into an 8-platform Enriched Extractor: direct CDN + views/likes for YouTube, TikTok, Instagram, X, Facebook, Reddit, Pinterest, Vimeo — batch 100, 512 MB, $3/1K. No yt-dlp. No ffmpeg. No 4GB RAM.

qbitlabs charges $0.05 per HD video ($50/1K) for 1000+ sites. easyapi does 1 URL/run at $2.99/1K + usage. rover bundles Cobalt + ffmpeg for 21 platforms but needs 4GB and 50 concurrent. We needed the opposite: one run, 100 URLs, enriched stats, no bytes stored.

Live on Apify: lexiie/all-in-one-media-extractor (expanded from instadl.ink) — Build 0.0.11 PPE+Usage $3.00/1,000 failed = free — YT dQw4w9WgXcQ view 1.8B in 366ms, IG reel Dcm8usVMMcG like 0 comment 397 without proxy.

The bill that pushed us off yt-dlp

We started like everyone else: wrapping instadl (IG + TikTok) with yt-dlp for YouTube. Local test was fine. On Apify it wasn’t:

Build 616MB vs 83MB: yt-dlp + ffmpeg base is 83MB, but 4K merge needs ffmpeg + 4GB RAM for 60s video. Our Node + undici Build 0.0.11 is 616MB including sharp for logo — yt-dlp version was ~900MB before we stripped it.
The math that kills PPE Only $3: 1K videos × 50MB avg = 50GB external transfer $0.2/GB = $10 + KVS 50GB store $0.05/1K writes + 20 CU ffmpeg $4 → cost $14 → profit = 0.8×$3 - $14 = -$12. That’s why qbitlabs must charge +$0.03/GB after 1GB and khadinakbar caps at 250MB. We didn’t want to charge for bytes we don’t need.
The workaround: Don’t download. Return temporary CDN (googlevideo expire ~4h, scontent 86.400s, fbcdn, tiktokcdn) and let browser → CDN or agent → CDN fetch when needed. For RAG/UGC you need the url + stats, not the mp4 in KVS.

That decision forced us HTTP-only. No yt-dlp binary, no Cobalt server, no ffmpeg.

What “enriched extractor” actually means (vs URL-only)

URL-only (burbn/easyapi) gives you https://...mp4 and stops. For RAG you then need 3 more scrapers for views/likes to rank, and for UGC you need follower counts to filter influencers.

We return one item with 3 blocks — metadata / stats / media (nested, pruned if undefined):

{
  "platform": "tiktok",
  "metadata": { "title": "Calm dog...", "uploader": "kings.deals", "duration": 12, "thumbnail": "https://..." },
  "stats": { "viewCount": 860465, "likeCount": 5047, "commentCount": 23, "shareCount": 167, "followers": 0 },
  "media": [{ "url": "https://v16m.tiktokcdn-us.com/...", "quality": "hd", "mimeType": "video/mp4", "width": 1080, "height": 1920 }]
}

Same shape for all 8:

YouTube dQw4w9WgXcQ → 8 qualities 2160p-720p 366ms → view 1.8B like 16M tags channelId
TikTok @moh.shofwan 7645528079783267605 → view 860K like 5K comment 23 share 167 followers 0 + mp4 HD/SD + mp3
Instagram Dcm8usVMMcG → like 0 comment 397 view 4 + caption payal_rajput_tv...
X SpaceX/status/184... → view/like/retweet
Facebook watch?v=101532... → HD/SD mp4 How to share...
Vimeo 22439234 The Mountain → progressive mp4 or dash + plays/likes
Reddit r/pics → fallback_url + ups 450 num_comments 23
Pinterest pin/112426... → orig image + repin

No second call. Dataset views/overview shows inputUrl, platform, type, webpageUrl — fields metadata/stats/media nested.

From instadl.ink to 8 platforms — what broke

I started with instadl.ink — my own fast, no-storage extractor for Instagram & TikTok. To make it monetizable on Apify, I expanded it to 8 platforms. Each new one broke at least once:

YouTube — lightweight innertube POST /youtubei/v1/player returns streamingData without signatureCipher (no decipher). WEB would need poToken.
TikTok — returns gzip without content-encoding header, so manual decompression + retry with new device.
Instagram — embed likeCount regex failed because HTML escaped → fix to permissive edge_liked_by pattern. view 4 is real from embed (null for many reels is normal).
X — syndication token. 404 for Nasa 174... because ID doesn’t exist, not code.
Facebook — login wall not logged in → PRIVATE_CONTENT free.
Vimeo — player.vimeo.com cert hijack in local ISP → ProxyAgent → origin ✅, but progressive empty for some (dash only) → fallback oembed.
Batch — 100 vs 1 (easyapi/qbitlabs 1 URL/run) — 5 concurrent 300ms cache 5m → 100 URLs ≈ 30s vs 100 runs 100× overhead.

Pricing that doesn’t punish batch

PPE+Usage ON → video-extracted $0.003 ($3/1K) event covers extractor, CU/proxy/transfer billed separate to user. failed/private = free (so IG PRIVATE_CONTENT doesn’t cost). PPE Only would be 0.8*3 - costs and we’d be -$12 for bytes.

10 URLs → Max $0.03 (set Minimum $0.01 in Store Monetization → Additional options so user can’t Max $0.001 free ride)
100 URLs 720p best → Max $0.30 + ~$1 compute (5 CU) + 0.1GB proxy $0.80 (only if RESIDENTIAL) → $1.30 vs qbitlabs $5 for HD ($0.05 per HD = $50/1K)
Discounts: BRONZE $0.003 → SILVER $0.0025 → GOLD $0.0015 — same easyapi tier, but we batch 100 so GOLD 100 URLs $0.15 cheaper.

Store now single event video-extracted PPE+Usage $3/1K. If you want 2 events basic $1.50 / enriched $3 later, just add enriched flag back.

What we intentionally don’t do (and why that’s a feature)

No private bypass — PRIVATE_CONTENT free, pass cookies only if you own the content. InstagramPrivSniffer (collab intended) and jatin-dot-py (polaris_timeline_connection 28% bypass, patched 16 Oct 2025) are ToS violations on Apify — Store would takedown.
No bytes — maxFileSizeMB just skips >250MB for agents; we don’t merge HLS. If you need bytes, use downloader with R2.
No 1000+ sites — qbitlabs 1000+ is yt-dlp list, we do 8 core social + enriched batch 100 without yt-dlp. For SoundCloud/Twitch we’ll add HTTP-only next, not yt-dlp.

Try the exact URLs we tested (no proxy needed for 3)

curl -X POST "https://api.apify.com/v2/acts/lexiie~all-in-one-media-extractor/runs?token=TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"urls":["https://www.youtube.com/watch?v=dQw4w9WgXcQ","https://www.tiktok.com/@moh.shofwan/video/7645528079783267605","https://www.instagram.com/reel/Dcm8usVMMcG/"],"quality":"720p","audioOnly":false}'

Dataset 0.0.11 0R4ECgZIOkkdbjVUb SUCCEEDED 512 MB apify/actor-node:20 returns metadata/stats/media nested — overview view shows inputUrl, platform, type, webpageUrl — latencyMs 366ms YT, 852ms IG, 2838ms TT (3 retries).

Storage inputUrl success platform webpageUrl metadata.title stats.viewCount media[0].url latencyMs — flat for dataset but clean for LLM RAG (no KVS file to fetch).

If you need 1000+ sites, use qbitlabs ($0.05 HD = $50/1K). If you need 8 core social + enriched + batch 100 + direct CDN without yt-dlp/ffmpeg/4GB, use this. Build 0.0.11 live at https://console.apify.com/actors/GdXcUau5zytXfvdvS — star it and request SoundCloud/Twitch next — we’ll add without yt-dlp.

DEV Community