DEV Community

Viktor
Viktor

Posted on

How to Scrape Whatnot with Node.js: Seller Stats, Live Shows, Search, and a Viewer Tracker (2026)

Whatnot is where a lot of trading cards, comics, Funko Pops, and sneakers get sold now, live, one auction at a time. If you sell there, or you're studying the market, you want numbers: which sellers move the most items, who is live right now, how many people are watching, and what a card goes for when it's listed.

The first thing most people search is "Whatnot API". There is one, but it probably isn't the one you want. So this post covers both: what the official Whatnot API gives you, and how to build a small Whatnot scraper in Node.js for the public data it doesn't cover.

The scraper is one file, no npm dependencies, Node 18+. It will:

  1. Discover seller handles from the sitemap Whatnot publishes in robots.txt
  2. Read a seller's profile: followers, items sold, rating, reviews, live status
  3. List a seller's live and scheduled shows, and their shop listings
  4. Run a search and read a category feed
  5. Read a single listing with price, condition, and seller
  6. Snapshot live viewers in a category every few minutes into a CSV

Everything below was run on October 28, 2026, and the output is real.

Before you start: Whatnot's Terms of Service prohibit scraping, crawling, and other automated access to the app. robots.txt allows the public pages used here, but robots rules and terms are different things. Treat this as a learning project or small-scale research, keep request rates very low, don't resell the data, and don't build profiles of individual buyers or sellers. If you're a seller who wants your own data, use the official API first.

Is there a Whatnot API?

Yes. Whatnot has an official Seller API (REST, https://api.whatnot.com/seller-api/rest/v2, docs at developers.whatnot.com). You create a personal access token in the developer dashboard and send it as Authorization: Bearer wn_pat_tk_.... It covers products, inventory, orders, shipments, your shows, and show analytics.

The catch is in the name. The Whatnot API is for your own store. It answers "what did I sell and what do I need to ship", not "how many followers does that seller have" or "who is live in PokΓ©mon cards right now". There is no official public endpoint for other sellers, search, or category feeds.

So, a quick decision table:

You want Use
Your own orders, inventory, show analytics Official Seller API
Other sellers' public stats, live shows, search results, listing prices The public website, which is what this Whatnot scraper reads

How whatnot.com looks from a scraper's point of view

Three findings shaped the code.

1. robots.txt is friendly and points to sitemaps. It allows everything except account pages, the player, and a few others (/account/, /dashboard, /messages, /order/, /player, /video/, ...). It also lists seller-sitemap.txt, users-sitemap-index.xml, and listings-sitemap-index.xml. That's your discovery layer.

2. The data is already in the HTML. whatnot.com is a Next.js app with Apollo GraphQL. When the server renders a page, it embeds the GraphQL results it used in two places:

  • (window[Symbol.for("ApolloSSRDataTransport")] ??= []).push({...}) script tags. These hold the query results for feeds: search, category, a seller's shows, a seller's shop.
  • self.__next_f.push([1,"..."]) script tags. That's the React Server Components payload, and it carries the seller profile and the listing details.

So you don't need a headless browser, and you don't need to call the GraphQL endpoint yourself. The browser's GraphQL calls go to /services/graphql, which is protected by Kasada bot detection. Leave it alone. The HTML is enough.

3. Cloudflare sits in front, and it cares about honesty. This surprised me. From Node's built-in fetch, a request that pretends to be Chrome gets Cloudflare's "Just a moment..." challenge every time. A request with an honest, descriptive User-Agent gets a normal 200:

round  user-agent                status  bytes   challenge?
0      Chrome 130 string         403     5768    true
0      whatnot-research-script   200     830788  false
0      (none, Node default)      403     5533    true
0      curl/8.5.0                403     5533    true
1      Chrome 130 string         403     5768    true
1      whatnot-research-script   200     830827  false
Enter fullscreen mode Exit fullscreen mode

My guess is that a Chrome User-Agent with a non-Chrome TLS fingerprint looks like a disguised bot, while a script that says it's a script looks like what it is. That might change, but "say who you are" is good practice anyway.

Step 1: A polite fetch helper

One request every 3 to 4 seconds, an honest User-Agent, 404 treated as "gone", and a hard back-off when Cloudflare shows up.

// whatnot.mjs - a small Whatnot scraper in Node.js (18+), no dependencies.
// Reads only public, robots-allowed pages: sitemaps, /user/, /tag/, /search, /listing/.
import fs from "node:fs";

const BASE = "https://www.whatnot.com";
// Say who you are. A fake Chrome UA from Node gets Cloudflare's challenge (see "Pitfalls").
const UA = process.env.WHATNOT_UA ?? "whatnot-research-script/1.0 (+mailto:you@example.com)";
const MIN_DELAY_MS = 3000; // one request every ~3 s is plenty for this kind of job

// ---------- Step 1: a polite fetch helper ----------

let lastRequest = 0;
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));

export class BlockedError extends Error {}

export async function get(url, { tries = 3 } = {}) {
  for (let attempt = 1; attempt <= tries; attempt++) {
    const wait = lastRequest + MIN_DELAY_MS + Math.random() * 1000 - Date.now();
    if (wait > 0) await sleep(wait);
    lastRequest = Date.now();

    const res = await fetch(url, {
      headers: { "user-agent": UA, "accept-language": "en-US,en;q=0.9", accept: "text/html,*/*" },
      redirect: "follow",
    });
    const body = await res.text();

    if (res.status === 404) return null;
    // Cloudflare's interstitial: 403 + "Just a moment..." (or a challenge platform script)
    if (res.status === 403 && /Just a moment|challenge-platform/.test(body)) {
      if (attempt === tries) throw new BlockedError(`Cloudflare challenge on ${url}`);
      await sleep(30_000 * attempt); // back off hard, don't hammer
      continue;
    }
    if (res.status === 429 || res.status >= 500) {
      await sleep(5_000 * attempt);
      continue;
    }
    if (!res.ok) throw new Error(`${res.status} on ${url}`);
    return body;
  }
  throw new Error(`gave up on ${url}`);
}
Enter fullscreen mode Exit fullscreen mode

BlockedError matters. If you see a challenge after a long back-off, stop. Don't rotate User-Agents or proxies to get around it.

Step 2: Read the data the page already ships

Two small parsers. The Apollo blocks are JavaScript object literals, not strict JSON: they contain undefined, so we swap it for null before JSON.parse. The RSC payload is a series of JSON strings that we concatenate. Then we pull out typed objects by looking for {"__typename":"..." and slicing a balanced {...}.

// ---------- Step 2: get the data the page already ships ----------

// (a) Apollo SSR transport: the GraphQL results the server rendered with.
// They are JS object literals (they contain `undefined`), not strict JSON.
const APOLLO = '(window[Symbol.for("ApolloSSRDataTransport")] ??= []).push(';

export function apolloResults(html) {
  const out = [];
  let i = 0;
  while ((i = html.indexOf(APOLLO, i)) !== -1) {
    const start = i + APOLLO.length;
    const end = html.indexOf(")</script>", start);
    const chunk = JSON.parse(html.slice(start, end).replace(/:undefined\b/g, ":null"));
    for (const v of Object.values(chunk.rehydrate ?? {})) if (v?.data) out.push(v.data);
    for (const ev of chunk.events ?? []) if (ev.type === "next" && ev.value?.data) out.push(ev.value.data);
    i = end;
  }
  return out;
}

// (b) React Server Components payload: self.__next_f.push([1,"..."]) strings.
export function rscText(html) {
  const re = /self\.__next_f\.push\(\[1,("(?:[^"\\]|\\.)*")\]\)/g;
  let m, text = "";
  while ((m = re.exec(html))) text += JSON.parse(m[1]);
  return text;
}

// Pull one balanced {...} object out of a text blob, starting at `from`.
function sliceObject(text, from) {
  let depth = 0, inStr = false, esc = false;
  for (let i = from; i < text.length; i++) {
    const c = text[i];
    if (inStr) {
      if (esc) esc = false;
      else if (c === "\\") esc = true;
      else if (c === '"') inStr = false;
    } else if (c === '"') inStr = true;
    else if (c === "{") depth++;
    else if (c === "}" && --depth === 0) return text.slice(from, i + 1);
  }
  return null;
}

// Every object in the RSC text that starts with {"__typename":"<type>"
export function rscObjects(text, typename) {
  const needle = `{"__typename":"${typename}"`;
  const out = [];
  let i = 0;
  while ((i = text.indexOf(needle, i)) !== -1) {
    const raw = sliceObject(text, i);
    if (raw) { try { out.push(JSON.parse(raw)); } catch { /* skip partial chunks */ } }
    i += needle.length;
  }
  return out;
}
Enter fullscreen mode Exit fullscreen mode

The brace slicer is a bit crude, but the payload is machine-generated JSON, so a string-aware depth counter is enough. If a slice fails to parse, we skip it.

Step 3: Find sellers from the sitemap

seller-sitemap.txt is a plain list of profile URLs. On October 28, 2026 it had 7,740 handles.

// ---------- Step 3: discover sellers from the sitemap robots.txt points to ----------

export async function sellerHandles() {
  const txt = await get(`${BASE}/seller-sitemap.txt`);
  return txt
    .split(/\r?\n/)
    .map((l) => l.trim().match(/\/user\/([^/?#]+)$/)?.[1])
    .filter(Boolean);
}
Enter fullscreen mode Exit fullscreen mode
$ node whatnot.mjs sellers 5
{
  "count": 7740,
  "sample": ["001spades", "1099comics", "10k_sportscards", "112comics", "123readycomics"]
}
Enter fullscreen mode Exit fullscreen mode

The sitemap is stale in places. In two random samples, 4 of 14 handles returned 404 (renamed or closed accounts), including 10k_sportscards from the sample above. That's normal. Treat 404 as data, not as an error.

Step 4: Seller profile and shows

A seller page renders whichever tab the seller picked as default. Some open on Shows, others on Shop, and the Apollo data changes with it (GetUserLiveStreams vs ProfileShop). Requesting /user/{handle}/shows forces the shows tab, so the data is always the same shape. The profile object (followers, sold count, rating) is in the RSC payload on every tab.

// ---------- Step 4: seller profile + shows from /user/{handle}/shows ----------

const money = (m) => (m ? { amount: m.amount / 100, currency: m.currency } : null); // amounts are in cents

function cleanShow(s) {
  return {
    id: s.id,
    title: s.title,
    status: s.status, // PLAYING = live now, CREATED = scheduled
    start_time: s.startTime ? new Date(s.startTime).toISOString() : null,
    active_viewers: s.activeViewers ?? null,
    bookmarks: s.totalWatchlistUsers ?? null,
    seller: s.user?.username ?? null,
    categories: (s.livestreamCategories ?? []).map((c) => c.label).join("|"),
    tags: (s.tags ?? []).map((t) => t.label).join("|"),
    url: `${BASE}/live/${s.id}`,
  };
}

export async function seller(handle) {
  // Plain /user/{handle} renders whichever tab the seller picked (shows or shop).
  // /shows forces the shows tab, so the Apollo data is always GetUserLiveStreams.
  const html = await get(`${BASE}/user/${encodeURIComponent(handle)}/shows`);
  if (!html) return null; // 404: renamed, banned, or deleted

  const user = rscObjects(rscText(html), "PublicUserNode").find(
    (u) => u.username?.toLowerCase() === handle.toLowerCase() && "followerCount" in u
  );
  if (!user) throw new Error(`no profile object on /user/${handle} (layout changed?)`);

  const streams = apolloResults(html).find((d) => d.getUser?.livestreams)?.getUser.livestreams;

  return {
    handle: user.username,
    id: Buffer.from(user.id, "base64").toString().split(":")[1] ?? user.id, // "PublicUserNode:190554"
    display_name: user.displayName?.trim() || null,
    bio: user.bio || null,
    followers: user.followerCount,
    following: user.followingCount,
    sold: user.soldCount, // null when the seller hides it / has none
    rating: user.sellerRating?.overall ?? null,
    reviews: user.sellerRating?.numReviews ?? 0,
    avg_ship_days: user.averageShipDays,
    is_live: user.isLive,
    verified: user.isVerifiedSeller,
    premier: !!user.premierShopStatus?.isPremierShop,
    shows_total: streams?.totalCount ?? null,
    shows: (streams?.edges ?? []).map((e) => cleanShow(e.node)), // live + scheduled, first 15
  };
}
Enter fullscreen mode Exit fullscreen mode

Two things in there are easy to miss:

  • IDs are base64 Relay IDs. UHVibGljVXNlck5vZGU6MTkwNTU0 decodes to PublicUserNode:190554. Keep the numeric part. It doesn't change when a seller renames their account, and the handle does.
  • Money is in cents. {"amount": 5500, "currency": "USD"} is $55.00. There's also a localizedPrice converted to your currency based on your IP (I got VND from Vietnam). Always store price, never localizedPrice.

A seller who was live while I ran this:

$ node whatnot.mjs seller hititfromthepack
{
  "handle": "hititfromthepack",
  "id": "25944732",
  "display_name": "HitItFromThePack",
  "followers": 16358,
  "following": 128,
  "sold": 4407,
  "rating": 4.9,
  "reviews": 753,
  "avg_ship_days": 1,
  "is_live": true,
  "verified": false,
  "premier": true,
  "shows_total": 7,
  "shows": [
    {
      "id": "359ac220-1a62-435d-9e2a-50c00ab85c4c",
      "title": "πŸ‡ΊπŸ‡Έ MY BIGGEST $1 START STREAM EVER! $100K+ MID ERA",
      "status": "PLAYING",
      "start_time": "2026-10-28T02:02:51.692Z",
      "active_viewers": 158,
      "bookmarks": 95,
      "categories": "PokΓ©mon Cards",
      "tags": "$1 Starts|Sudden Death|Vintage",
      ...
    },
    {
      "id": "a1854a1a-e238-421c-b057-99162b365ae5",
      "status": "CREATED",
      "start_time": "2026-10-29T19:38:00.000Z",
      "active_viewers": 0,
      "bookmarks": 58,
      ...
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode

PLAYING means live now, CREATED means scheduled. The seven shows were one live and six upcoming, so the profile shows live and scheduled streams, not the archive. bookmarks (totalWatchlistUsers) on a scheduled show is a nice early signal of how big that show will be.

And a random sample from the sitemap, via node whatnot.mjs sellers-csv 6:

handle,followers,sold,rating,reviews,avg_ship_days,is_live,premier,shows_total
sundazebest,7839,13691,4.9,2878,,false,false,0
comicselitelive,25574,91897,5,17060,0,true,false,51
king_espeo,197,117,5,43,,false,false,0
bfunko_collectibles,1262,414,4.8,149,,false,false,0
Enter fullscreen mode Exit fullscreen mode

(The other two of the six were 404s.) sold is lifetime items sold, which is the most useful single number for sizing a seller. avg_ship_days is often null.

The seller's shop

Many sellers also keep buy-it-now and auction listings in a shop. /user/{handle}/shop ships the first 20 of them:

export async function shop(handle) {
  const html = await get(`${BASE}/user/${encodeURIComponent(handle)}/shop`);
  if (!html) return null;
  const feed = apolloResults(html).find((d) => d.getUser?.searchShop)?.getUser.searchShop;
  return flattenFeed(feed).listings; // first 20 listings
}
Enter fullscreen mode Exit fullscreen mode
$ node whatnot.mjs shop bangarangcards
[
  { "title": "Typhoid Mary", "price": 5, "currency": "USD", "type": "BUY_IT_NOW", "status": "ACTIVE", ... },
  { "title": "Thena", "price": 15, "currency": "USD", "type": "BUY_IT_NOW", "status": "ACTIVE", ... },
  ...
]
Enter fullscreen mode Exit fullscreen mode

Step 5: Search and category feeds

Search (/search?query=...) and category pages (/tag/{name}) both render a Feed. Its edges are a mix: FeedEntity wrappers around listings or livestreams, plus Section carousels. We flatten them and keep the three types we care about.

// ---------- Step 5: search and category feeds ----------

function cleanListing(l) {
  return {
    id: l.id,
    title: l.title,
    subtitle: l.subtitle ?? null,
    price: money(l.price)?.amount ?? null,
    currency: l.price?.currency ?? null,
    type: l.transactionType, // BUY_IT_NOW or AUCTION
    status: l.listingStatus ?? l.publicStatus,
    current_bid: money(l.currentBid)?.amount ?? null,
    bids: l.currentBidCount ?? null,
    quantity: l.quantity,
    seller: l.user?.username ?? null,
    url: `${BASE}/listing/${encodeURIComponent(l.id)}`,
  };
}

// Feeds mix listings, livestreams, users and "Section" carousels. Flatten them.
function flattenFeed(feed) {
  const shows = [], listings = [], users = [];
  const visit = (o) => {
    if (!o) return;
    if (o.__typename === "LiveStream") shows.push(cleanShow(o));
    else if (o.__typename === "ListingNode") listings.push(cleanListing(o));
    else if (o.__typename === "PublicUserNode") users.push({ handle: o.username, followers: o.followerCount, sold: o.soldCount });
  };
  for (const { node } of feed?.objects?.edges ?? []) {
    if (node.__typename === "FeedEntity") visit(node.object);
    else visit(node);
  }
  const dedupe = (arr) => [...new Map(arr.map((x) => [x.id ?? x.handle, x])).values()];
  return { title: feed?.title, hasNextPage: feed?.objects?.pageInfo?.hasNextPage ?? false,
    shows: dedupe(shows), listings: dedupe(listings), users: dedupe(users) };
}

export async function search(query) {
  const html = await get(`${BASE}/search?query=${encodeURIComponent(query)}&referringSource=typed`);
  const feed = apolloResults(html).find((d) => d.search?.objects)?.search;
  return flattenFeed(feed);
}

export async function category(tag) {
  const html = await get(`${BASE}/tag/${encodeURIComponent(tag)}`);
  if (!html) return null;
  const feed = apolloResults(html).find((d) => d.feed?.objects)?.feed;
  return flattenFeed(feed);
}
Enter fullscreen mode Exit fullscreen mode
$ node whatnot.mjs search charizard
# 18 listings, hasNextPage: true
{ "title": "Charizard Expedition Reverse Holo", "subtitle": "Moderately Played", "price": 175, "currency": "USD", "seller": "bigmacspokeshack", ... }
{ "title": "Charizard ex Special Collection", "subtitle": "New βˆ™ Journey Together", "price": 50, "currency": "USD", "seller": "takeflightcards", ... }

$ node whatnot.mjs category pokemon_cards
# "PokΓ©mon Cards": 24 shows, all PLAYING
{ "seller": "hititfromthepack", "active_viewers": 160, "bookmarks": 95, "title": "πŸ‡ΊπŸ‡Έ MY BIGGEST $1 START STREAM EVER! ..." }
{ "seller": "koheitcg_jp", "active_viewers": 22, "tags": "PokΓ©mon|Pokemon 30th Anniversary|$1 Starts|Shop from Japan|...", ... }
Enter fullscreen mode Exit fullscreen mode

The limit: you only get the first page. Search gives about 18 to 24 items, and a category gives 24 shows. Later pages and the vertical filters (shows only, sellers only) are loaded by the browser through the Kasada-protected GraphQL endpoint. Adding &vertical=LIVESTREAM to the URL did nothing in my tests. For monitoring "what's big right now" the first page is fine. For a full catalog it isn't.

Also, the category feed is ranked, not sorted by viewers. Sort it yourself.

Step 6: A single listing

Listing pages carry the full ListingNode in the RSC payload, including description, created date, condition and other attributes. The node doesn't include its seller. The page passes the seller as a separate "seller":{...} prop, so we grab that too.

// ---------- Step 6: a single listing ----------

export async function listing(idOrUrl) {
  const id = decodeURIComponent(idOrUrl.split("/listing/").pop());
  const html = await get(`${BASE}/listing/${encodeURIComponent(id)}`);
  if (!html) return null;
  const text = rscText(html);
  const node = rscObjects(text, "ListingNode").find((l) => l.id === id);
  if (!node) return null;
  // The listing node itself has no user; the page passes the seller as a separate prop.
  const at = text.indexOf('"seller":{"__typename":"PublicUserNode"');
  const sellerNode = at === -1 ? null : JSON.parse(sliceObject(text, at + '"seller":'.length));
  const attrs = Object.fromEntries((node.listingAttributeValues ?? []).map((a) => [a.attribute.label, a.value]));
  return { ...cleanListing(node), seller: sellerNode?.username ?? null, description: node.description, created_at: node.createdAt,
    offers_allowed: node.transactionProps?.isOfferable ?? null, attributes: attrs };
}
Enter fullscreen mode Exit fullscreen mode
$ node whatnot.mjs listing "https://www.whatnot.com/listing/TGlzdGluZ05vZGU6MjQ0ODA3NTg0MQ=="
{
  "id": "TGlzdGluZ05vZGU6MjQ0ODA3NTg0MQ==",
  "title": "Charizard Expedition Reverse Holo",
  "subtitle": "Moderately Played",
  "price": 175,
  "currency": "USD",
  "type": "BUY_IT_NOW",
  "status": "ACTIVE",
  "quantity": 1,
  "seller": "bigmacspokeshack",
  "description": "Charizard Expedition Reverse Holo\nMP",
  "created_at": "Fri, 16 Oct 2026 20:25:26 GMT",
  "offers_allowed": true,
  "attributes": { "Condition": "Moderately Played" }
}
Enter fullscreen mode Exit fullscreen mode

Listing IDs are base64 too (ListingNode:2448075841), and they can end in =. Encode them when you build URLs.

Step 7: A live-viewer tracker

Now the fun part. Live shopping is all about who's live and how many people are watching. Snapshot a category every few minutes and append each live show to a CSV:

// ---------- Step 7: live-viewer tracker ----------

export const csvCell = (v) => (v == null ? "" : /[",\n]/.test(String(v)) ? `"${String(v).replace(/"/g, '""')}"` : String(v));

export async function track(tag, { every = 10, times = 3, file = "live-snapshots.csv" } = {}) {
  const cols = ["snapshot_at", "tag", "show_id", "seller", "status", "active_viewers", "bookmarks", "start_time", "title"];
  if (!fs.existsSync(file)) fs.writeFileSync(file, cols.join(",") + "\n");
  for (let n = 1; n <= times; n++) {
    const at = new Date().toISOString();
    const feed = await category(tag);
    const live = (feed?.shows ?? []).filter((s) => s.status === "PLAYING");
    const rows = live.map((s) => [at, tag, s.id, s.seller, s.status, s.active_viewers, s.bookmarks, s.start_time, s.title]);
    fs.appendFileSync(file, rows.map((r) => r.map(csvCell).join(",")).join("\n") + (rows.length ? "\n" : ""));
    const viewers = live.reduce((a, s) => a + (s.active_viewers ?? 0), 0);
    console.error(`[${at}] ${tag}: ${live.length} live shows, ${viewers} viewers in the first feed page`);
    if (n < times) await sleep(every * 60_000);
  }
}
Enter fullscreen mode Exit fullscreen mode

I ran it on PokΓ©mon cards, every 2 minutes, 3 times:

$ node whatnot.mjs track pokemon_cards 2 3
[2026-10-28T04:12:14.735Z] pokemon_cards: 24 live shows, 6096 viewers in the first feed page
[2026-10-28T04:14:15.576Z] pokemon_cards: 24 live shows, 6150 viewers in the first feed page
[2026-10-28T04:16:16.457Z] pokemon_cards: 24 live shows, 7125 viewers in the first feed page
Enter fullscreen mode Exit fullscreen mode
snapshot_at,tag,show_id,seller,status,active_viewers,bookmarks,start_time,title
2026-10-28T04:12:14.735Z,pokemon_cards,359ac220-...,hititfromthepack,PLAYING,155,95,2026-10-28T02:02:51.692Z,πŸ‡ΊπŸ‡Έ MY BIGGEST $1 START ...
2026-10-28T04:12:14.735Z,pokemon_cards,729592e6-...,koheitcg_jp,PLAYING,24,21,2026-10-28T03:00:17.480Z,πŸ‡―πŸ‡΅ Superbly Spectacular ...
2026-10-28T04:12:14.735Z,pokemon_cards,e3538101-...,clovrcards,PLAYING,208,152,2026-10-27T19:04:20.600Z,πŸ‡ΊπŸ‡Έ πŸ‰ EX EMERALD BOX ...
Enter fullscreen mode Exit fullscreen mode

Grouping by seller across the three snapshots already tells a story:

spacenarwhalz      2043 β†’ 1993 β†’ 1963
novatcg            1075 β†’ 1137 β†’ 1252
saw_tcg             160 β†’  154 β†’ 1096
shop_exp            531 β†’  551 β†’  529
Enter fullscreen mode Exit fullscreen mode

A jump like saw_tcg's is exactly what you'd want an alert for. 27 different sellers showed up, and 22 of them were in all three snapshots. The other five moved in or out of the first page as the ranking shifted. That churn is a property of the feed, not of the shows, so don't read a missing row as "the show ended". To confirm that, check the seller's /shows page.

For a real tracker, run it from cron every 10 to 15 minutes, not every 2. That's 4 to 6 requests an hour per category, which is about as gentle as it gets.

Step 8: The CLI

A tiny command switch ties it together. Paste the blocks above into one whatnot.mjs (Node 18+, ES module, no dependencies), add this at the bottom, and run node whatnot.mjs for usage.

// ---------- CLI ----------

async function main() {
  const [cmd, ...args] = process.argv.slice(2);
  const print = (x) => console.log(JSON.stringify(x, null, 2));
  switch (cmd) {
    case "sellers": {
      const handles = await sellerHandles();
      print({ count: handles.length, sample: handles.slice(0, Number(args[0] ?? 10)) });
      break;
    }
    case "sellers-csv": {
      // Profile a random sample of sitemap sellers into sellers.csv
      const n = Number(args[0] ?? 10);
      const handles = (await sellerHandles()).sort(() => Math.random() - 0.5).slice(0, n);
      const cols = ["handle", "followers", "sold", "rating", "reviews", "avg_ship_days", "is_live", "premier", "shows_total"];
      const lines = [cols.join(",")];
      for (const h of handles) {
        const s = await seller(h).catch((e) => { if (e instanceof BlockedError) throw e; return null; });
        if (s) lines.push(cols.map((c) => csvCell(s[c])).join(","));
        console.error(`${h}: ${s ? `${s.followers} followers, ${s.sold ?? "?"} sold` : "skipped"}`);
      }
      fs.writeFileSync("sellers.csv", lines.join("\n") + "\n");
      break;
    }
    case "seller":
      for (const h of args) {
        try { print(await seller(h) ?? { handle: h, error: "not found" }); }
        catch (e) { print({ handle: h, error: e.message }); if (e instanceof BlockedError) return; }
      }
      break;
    case "shop": print(await shop(args[0])); break;
    case "search": print(await search(args.join(" "))); break;
    case "category": print(await category(args[0])); break;
    case "listing": print(await listing(args[0])); break;
    case "track": await track(args[0], { every: Number(args[1] ?? 10), times: Number(args[2] ?? 3) }); break;
    default:
      console.log("usage: node whatnot.mjs sellers [n] | sellers-csv [n] | seller <handle...> | shop <handle> | search <query> | category <tag> | listing <id|url> | track <tag> [everyMin] [times]");
  }
}

if (import.meta.url === `file://${process.argv[1]}`) main().catch((e) => { console.error(e.message); process.exit(1); });
Enter fullscreen mode Exit fullscreen mode

Pitfalls, in one place

  • Terms of Service. Whatnot's terms prohibit automated access. robots.txt allowing a path doesn't change that. Keep it small, slow, and non-commercial, or use the official Seller API for your own data.
  • Cloudflare checks if your client is consistent. A fake browser UA from Node got challenged every time. An honest UA passed. If you get challenged anyway, back off and stop.
  • GraphQL is off-limits in practice. The browser's calls go through Kasada-protected /services/graphql, and introspection is disabled. Everything here comes from server-rendered HTML.
  • First page only. Search and category pages give roughly one page of results (about 24). No deep pagination without the browser's GraphQL calls.
  • Default tabs differ per seller. Use /user/{handle}/shows and /user/{handle}/shop explicitly.
  • Money is in cents, and localizedPrice follows your IP. Store price.amount / 100 with price.currency.
  • IDs are base64 Relay IDs. Decode user IDs and key on the number. Handles change.
  • Sitemaps are stale. Expect a share of 404s (4 of 14 in my samples).
  • Feeds are ranked and churn. Sort by viewers yourself, and don't infer "ended" from absence.
  • The markup will change. The parsers key on __typename and Apollo query results, which are more stable than CSS classes. If seller() throws "no profile object", look at the page again.

When DIY stops being worth it

The single-file scraper is fine for a few sellers and a category or two. It gets painful when you need page 2+ of search, a seller's full show history and reviews, or thousands of profiles a day without tripping Cloudflare.

For that I use ScrapeAtlas*, which has Whatnot endpoints with real cursor pagination. GET /v1/whatnot/search takes a query, a vertical (UNIVERSAL, LIVESTREAM, PRODUCT, USER, or CATEGORY), and a cursor, and returns 24 records per page. /v1/whatnot/profile takes a handle, user_id, or url, and there are /v1/whatnot/user/shows, /user/listings, /user/reviews, and /user/clips per seller, plus /v1/whatnot/category, /livestream, /livestream/listings, /listing, /product, and /categories. Auth is an x-api-key header, and failed requests don't cost credits.

If you only need what's on the first page, though, the DIY route above works fine.

Wrapping up

  • The official Whatnot API is a Seller API. Great for your own store, no help for market data.
  • The data is in the HTML. Apollo SSR blocks for feeds, the RSC payload for profiles and listings. No headless browser needed.
  • Be honest and slow. An honest User-Agent passed Cloudflare where a fake Chrome one didn't. Keep to one request every few seconds.
  • Know the limits. First page only, ranked feeds, prices in cents, base64 IDs, stale sitemaps.
  • Live viewers are the interesting metric. A tiny cron job gives you a category-level view of who's pulling a crowd.

What would you track on Whatnot: seller growth, prices for one card, or live viewers by category? Tell me in the comments.


*I build it.

Top comments (0)