DEV Community

Devil Scrapes
Devil Scrapes

Posted on

Per-profile or per-link: the billing unit on a link-in-bio scraper differs by 10x

Quick answer

On a link-in-bio page, "one row per profile" and "one row per link" aren't a formatting preference — they're two different products, and they diverge by roughly 10x on a typical creator profile. A profile with 68 links bills as 1 row under the first model and 68 rows under the second. The Linktree Profile Scraper bills per link, on purpose: the link is the unit you actually analyze — "where does this creator send traffic" — not the profile.

Why does the billing unit matter more than the price-per-result headline?

Because "price per result" is meaningless until you know what a result is. A scraper billing $1/1,000 profiles and a scraper billing $4/1,000 links look like the first is 4x cheaper — until you fetch the same 68-link profile from both and realize the first one gave you one flat JSON blob you'll spend engineering time re-parsing into link-level rows yourself, while the second one already gave you 68 clean, typed, filterable rows. The per-profile scraper's "1,000 results" is 1,000 profiles, which could be tens of thousands of underlying links; the per-link scraper's "1,000 results" is 1,000 analyzable facts. Compare the cost per useful fact, not the cost per row, and the ranking can flip entirely.

We chose per-link billing for this Actor because a link-in-bio page's value is almost entirely in its outbound destinations — link type, position, group membership, gate status — not in the container profile around them. link_url, link_type, position, group_id/group_title, and gated/gate_type are all first-class columns precisely because those are the fields a link-in-bio analysis actually filters and joins on. Denormalizing the profile identity fields (username, profile_title, profile_is_verified, profile_tier) onto every link row means you never have to join back to a separate profile table to answer "which verified creators link to a specific commerce product."

What's the second finding, below the billing question?

Linktree's __NEXT_DATA__ payload — the JSON blob the page ships instead of server-rendered HTML — carries a per-link rules.gate object: age, passcode, password, or NFT. That means some links inside a single profile's payload are ones a visitor literally cannot click through without clearing a gate, and the payload doesn't mark that at the top level — it's per link, buried in rules. A scraper that emits link_url and link_title without reading rules.gate hands you a row that looks exactly like every other clickable link and isn't one. We surface that as gated (boolean) and gate_type (which gate) on every row, without attempting to clear the gate — you get an accurate picture of what a real visitor can and can't reach, not a false completeness count.

Before comparing "price per result" across two scrapers on the same target, check what a result is. A cheaper price on a coarser unit can cost more per fact you actually need, and it's the buyer's job to check the unit — a listing rarely states it plainly.

What the Actor gives you

One row per link (not per profile): link_id, link_title, link_url, link_type (CLASSIC, GROUP, YOUTUBE_VIDEO, COMMERCE_PRODUCT, and any newer type Linktree adds — never dropped), position, group_id/group_title for child-of-group links, thumbnail_url, gated/gate_type, meta_title/meta_description, and scraped_at. Every row also carries the parent profile's profile_title, profile_description, profile_avatar_url, profile_is_verified, profile_tier, and profile_has_password, denormalized so you never need a join to answer a profile-level question from a link-level table.

Honest limitations 🚧

Public profiles only — no login-required data, analytics, or edit-mode fields. Gated links are flagged, never bypassed; this Actor tells you a gate exists, it doesn't try to get past one. Group membership is a flat group_id/group_title join, not a nested tree — reconstruct hierarchy yourself if you need it. Zero-link profiles emit no rows for that profile (logged, not a failure); the run only fails if every requested profile yields zero rows.

FAQ

Why does this bill per link instead of per profile?
Because the link is the analyzable unit on a link-in-bio page — you're asking "where does this account send traffic," and a per-profile price hides how many or how few links you actually got for it.

Does this need a Linktree account or API key?
No — it reads the same public __NEXT_DATA__ JSON payload the profile page itself serves.

What happens to age/passcode/password/NFT-gated links?
They're returned with gated: true and the specific gate_type set, but the gate itself is never cleared — you know a link exists and is protected, without us attempting to see behind it.

At 1,000 link rows, that's $4.20 / 1,000 ($0.20 start + $4.00 for the rows).

→ Linktree Profile Scraper on Apify


Built by Devil Scrapes. We rotate browser fingerprints and keep the dataset clean — Pydantic-validated, ISO-8601 timestamps — so the unit you're billed on is exactly the unit you can build on.

Top comments (0)