I Crawled 554 SaaS Affiliate Pages. My Parser Lied to Me Nine Times.

Tien Luong — Fri, 30 Oct 2026 09:14:38 +0000

I wanted to know which SaaS companies actually pay affiliates well. Not "best affiliate programs 2026" listicles — those are themselves affiliate pages, written by people who get paid when you click. I wanted the terms straight off the vendors' own sites.

So I wrote a crawler. 554 domains in, 151 programs out, 22 minutes.

Here's the headline number: of 151 programs, only 17 pay lifetime commission — and 58 of them don't disclose the payout structure at all.

Correction, 30 October 2026. This post first reported 19 lifetime programs. I have since hand-checked all 19 of those pages and found two misclassified: Omnisend states a 24-month cap in the same sentence as its lifetime clause, and Woodpecker's affiliate paths all return 200 while redirecting to its homepage — a soft 404 my crawler had no defence against. The correct figure is 17. The parser bug behind the Omnisend case is fixed and covered by tests. Every figure below is updated; the original numbers are shown where the change matters. Of those 17, only 9 say lifetime without a condition elsewhere on the page walking it back — that check is its own post.

But that number took four full rewrites to trust, because my parser was confidently wrong nine separate times. The bugs are the interesting part, so that's most of what this post is about.

The build

Node 26, standard library only. No cheerio, no puppeteer, no axios.

node:sqlite for storage (DatabaseSync is stable now — no better-sqlite3 needed)
global fetch with AbortController for timeouts
regex for HTML, which I'll defend in a second

The flow per domain:

Fetch /robots.txt, parse the User-agent: * block, honor every Disallow.
Fetch the homepage, scan <a> tags for anything matching affiliate|partner program|referral.
Probe 14 conventional paths: /affiliates, /affiliate-program, /partners/affiliate, /become-an-affiliate, and so on.
Score every candidate URL, fetch the best 8, extract terms.

Politeness: 5 concurrent domains, 700ms between hits on the same host, 20s timeout, honest User-Agent with the tool name in it.

On regex-for-HTML: I'm not parsing a document tree. I'm looking for the sentence "Earn 30% recurring commission" in a 460KB React-rendered blob where the DOM structure carries no meaning — the number and the word recurring are nine <span>s apart for styling reasons. Strip tags, collapse whitespace, pattern-match the prose. A real parser would buy me nothing here.

function bocThe(html) {
  return html
    .replace(/<script[\s\S]*?<\/script>/gi, ' ')
    .replace(/<style[\s\S]*?<\/style>/gi, ' ')
    .replace(/<noscript[\s\S]*?<\/noscript>/gi, ' ')
    .replace(/<!--[\s\S]*?-->/g, ' ')
    .replace(/<[^>]+>/g, ' ')
    .replace(/&nbsp;/gi, ' ')
    .replace(/&amp;/gi, '&')
    .replace(/&#36;/g, '$')
    .replace(/\s+/g, ' ')
    .trim();
}

That's the whole HTML layer.

Lie #1: It confused "sells affiliate software" with "pays commission"

First run, Systeme.io came back at 30%. The real number is 60%.

The crawler had landed on systeme.io/affiliate-management — Systeme's product page for businesses who want to run their own affiliate program. The actual affiliate program lives at /affiliate-program.

Both pages match /affiliate/. Both are full of the word commission. One pays you; one sells you software for paying other people.

Two fixes. First, score the URL shape:

const URL_TOT = [            // good
  [/\/affiliate[-_]program\b/i, 10],
  [/\/become[-_]an[-_]affiliate\b/i, 10],
  [/\/affiliates?\/?(\?|#|$)/i, 9],
  [/\/partner[-_]program\b/i, 7],
];

const URL_XAU = [            // bad
  [/affiliate[-_](management|software|platform|tracking|tool)/i, -25],
  [/\/(blog|resources|guides?|articles?|academy)\//i, -18],
  [/\/(features?|products?|solutions?|integrations?)\//i, -12],
  [/\/pricing\b|\/compare\b|\/vs[-_]/i, -10],
];

Second — and this one caught cases the URL didn't — read which side of the transaction the page is addressing:

const BEN_THAM_GIA = /join (?:our|the) affiliate|become an affiliate|you(?:'ll| will)? earn|refer[^.]{0,30}and earn|start (?:promoting|earning)/gi;

const BEN_BAN_PHAN_MEM = /your affiliates\b|manage (?:your )?affiliates|affiliate (?:software|platform|management)|launch your (?:own )?affiliate|recruit affiliates/gi;

Count both. If the sell-side wins, discard the page. "you'll earn" and "your affiliates" are the same topic pointed in opposite directions, and that single distinction fixed a whole class of false positives.

Lie #2: It counted competitor names as the network in use

That same Systeme page name-drops PartnerStack and Tapfiliate — as competitors, in a comparison table.

My network detector was running regex over the full page text, so it tagged Systeme as running on both.

// Before: /partnerstack/i.test(text)
// After:
function doMang(html) {
  const lien = [...html.matchAll(/(?:href|src|action)\s*=\s*["']([^"']+)["']/gi)]
    .map((m) => m[1]).join(' ');
  return MANG.filter(([re]) => re.test(lien)).map(([, ten]) => ten);
}

A company that runs on PartnerStack has a partnerstack.com URL in an href, a tracking script, or a form action. A company that merely mentions PartnerStack has it in a paragraph. Scan attribute values, not prose.

Lie #3: It read the cookie duration backwards

Systeme's terms table says, verbatim:

Lifetime, referral tagged to you permanently, not just for 30/60 days

My parser recorded: cookie 30 days.

It found 30, it found days, it was adjacent — correct pattern, exactly inverted meaning. The number is there precisely to be negated.

function trichCookie(text) {
  // Check lifetime FIRST. Pages write "permanent, not just for 30/60 days" —
  // grabbing the number inverts the meaning.
  if (/lifetime[^.]{0,30}(?:cookie|attribution)|never expires?|tagged to you permanently/i.test(text))
    return 9999;

  const m = text.match(/(\d{1,3})[\s-]*day[s]?[\s-]*(?:tracking\s*)?cookie/i);
  if (!m) return null;

  const quanh = text.slice(Math.max(0, m.index - 60), m.index + 60);
  if (/not just|rather than|unlike|instead of|more than/i.test(quanh)) return null;

  return parseInt(m[1], 10);
}

Two rules, and they generalize: check the strong claim before the weak one, and look at what surrounds a number before believing it. A regex match tells you a pattern occurred. It tells you nothing about whether the sentence was affirming or denying it.

Lie #4: It grabbed prices, mockups, and totals as per-sale payouts

This is where the output got funny.

Vendor	Parser said	What it actually read
Writesonic	$23,940 per sale	A number in a marketing screenshot of an earnings dashboard
make.com	$1,433.04	A one-year total for 10 referrals
hunter.io	$297.30	Total for 10 referrals — real rate is $27.93
systeme.io	$970	The annual plan price

Four different failure modes, one root cause: a $ followed by digits is not a commission. It's a dollar amount on a page that contains many dollar amounts, most of which are prices.

The fix was to require the payout unit to be adjacent:

const DON_VI = '(sale|referral|customer|signup|conversion|subscriber|client|order)';

const mau = [
  new RegExp('\\$\\s?([\\d,]+(?:\\.\\d{1,2})?)\\s*(?:USD\\s*)?(?:per|\\/|for each|for every)\\s*(?:new\\s+|qualified\\s+|paying\\s+)?' + DON_VI, 'gi'),
  new RegExp('(?:earn|get|receive|pay(?:s|ing)?|up to)\\s*\\$\\s?([\\d,]+(?:\\.\\d{1,2})?)\\s*(?:USD\\s*)?(?:per|for each|for every)\\s+(?:new\\s+|qualified\\s+|paying\\s+)?' + DON_VI, 'gi'),
];

// ...and a bound the first version simply did not have:
if (!(tien > 0) || tien > 10000) continue;

Plus explicit rejections for /year, /month, refer \d+, from \d+ users, this year — the grammar of an aggregate rather than a unit rate.

$23,940 should have been impossible. I had no sanity bound on the output at all. A parser that can emit a number four orders of magnitude outside the plausible range is a parser with no range.

Lie #5: The word "lifetime" elsewhere on the page overrode the real term

Kartra's page, live right now:

Up to 40% recurring commissions for the first 12 months per qualified referral.

And in its own FAQ, unambiguously:

Do you pay lifetime commissions? We do not offer lifetime commissions.

My parser classified Kartra as lifetime. That's a 30-point swing in the ranking — it would have sat near the top of the list on a term it explicitly denies offering.

The word lifetime appeared somewhere on the page. My classifier checked the lifetime pattern before the duration pattern, and returned on first match.

The fix is ordering, and it's the opposite of lie #3's fix — which is the real lesson:

// "first N months" beats "lifetime". A concrete duration is a harder
// commitment than a vague adjective, so it wins.
const m = n.match(COTHANG);
if (m) return { kieu: 'định kỳ có hạn', thang: parseInt(m[1] || m[2] || m[3], 10) };

if (TRONDOI.test(n)) return { kieu: 'trọn đời', thang: null };

For cookies, lifetime wins because the number is usually being negated. For commission terms, the number wins because the adjective is usually marketing. There's no universal precedence rule. You have to know what the page is doing with each field.

Related: snov.io writes "Get 1 year lifetime commission" — self-contradictory in four words. I take the concrete duration: 12 months.

Lie #6: Earnings calculators poisoned everything

lemlist's page says:

Earn a 25% commissions for 12 months for each lemlist license purchased

My parser recorded 2 months.

The page has an interactive earnings calculator — a slider for referral count, a monthly price, projected totals. That widget's numbers are in the DOM. So are the pricing table's, the footer's, and the testimonials'.

I was scanning the entire page for /(\d+)\s*months?/ and taking what I found.

function vungHoaHong(text) {
  // Scan ONLY ±130 chars around commission keywords. A whole-page scan
  // picks up earnings calculators, pricing tables, footers.
  const doan = [];
  const re = /commission|recurring|you earn|payout|rev\s*share/gi;
  let m;
  while ((m = re.exec(text)) && doan.length < 40) {
    doan.push(text.slice(Math.max(0, m.index - 130), m.index + 130));
  }
  return doan.join(' ||| ');
}

Every extraction now runs against these windows, never the raw page. beehiiv had the same bug — "3 months" from an unrelated widget.

Lie #7: A percentage glued to the wrong noun

swipepages.com, in a feature banner:

Upto 30% commission · 60 day cookie lifespan · 100% timely payouts

Parser said 100%. Real rate: 30%.

The 100% belongs to timely payouts — a boast about paying on schedule. It isn't a commission rate at all. My context window was ±110 characters and the word commission sat right there in the same banner, so the commission-word gate passed happily.

Proximity said yes. Grammar said no. I was only checking proximity.

// Nouns that commonly follow a percentage WITHOUT it being a commission.
const NGAY_SAU_XAU = /^\s*(?:timely|on[-\s]?time|secure|safe|accurate|free|money[-\s]?back|refund|satisfaction|guarantee[d]?|uptime|automated|transparent|verified|retention|of (?:our )?(?:users|customers|clients))\b/i;

const ngaySau = text.slice(m.index + m[0].length, m.index + m[0].length + 40);
if (NGAY_SAU_XAU.test(ngaySau)) continue;

Marketing banners line up unrelated numbers for visual rhythm: 30% commission, 60 day cookie, 100% timely payouts. Three different units in one row, with no grammar tying any of them together. A window-based extractor sees one soup of tokens.

Lie #8: Lie #3 again, pointed the other way

fliki.ai:

30-day cookie · 30% lifetime commission

Parser said permanent cookie. Real: 30 days.

My lifetime-cookie pattern allowed 30 characters of slack between the two words:

/(?:cookie|attribution)[^.]{0,30}lifetime/i

And "cookie 30% lifetime" fits in that gap perfectly. But the lifetime modifies commission, not cookie — the cookie is stated as 30 days two words earlier.

This is bug #3 inverted, in the same field, in the same regex. #3 missed a permanent cookie because it trusted a number that was being negated. #8 invented a permanent cookie because it trusted an adjective that belonged to a different noun. One regex, two opposite ways to be wrong.

// No % between the two words, and "lifetime" must not be modifying commission.
/(?:cookie|attribution)[^.%]{0,25}lifetime(?!\s*(?:recurring|commission|payout|revenue|rev\s*share))/i

Here's the part that bothers me. Neither of these was caught by a test. They were caught because the output was absurd — two programs claiming 100% commission paid forever. No business survives paying out 100% of revenue in perpetuity, so I went and read the pages.

Absurdity is a weak detector. It catches $23,940 per sale and 100% forever. It does nothing about a program that pays 25% when the page says 20%. Every wrong-but-plausible number my parser has ever emitted is still sitting in that database, and I have no way to find them except by hand.

Lie #9: Attributes merged across two different sentences

This one I found by hand, and only because the number was absurd.

purevpn.com's page contains these two statements, in different sentences, in different sections:

Our monthly plan brings you a 100% flat commission, while all other plans get 40%

Flat 35% recurring commission for lifetime on all plans.

My parser reported: 100%, lifetime.

That program does not exist. The 100% is a one-time bounty on the monthly plan. The lifetime term pays 35%. The parser took the percentage from sentence A and the payout type from sentence B and stapled them together into terms no page ever offered.

The cause was my own fix from lie #6. I had narrowed extraction to ±130-character windows around commission keywords — which solved the earnings-calculator problem, but a 130-character window still straddles sentence boundaries. Inside one window, two unrelated claims look like one claim.

My first fix was to make the sentence the unit of extraction instead of the window. That failed, and the way it failed is the interesting part.

The real page text is a single sentence:

Our monthly plan brings you a 100% flat commission, while all other plans get you 40%, on top of it a 35% recurring commission for lifetime on all plans.

Three different commission rates, comma-separated, one sentence. Sentence scope put all three in the same bucket and the largest won again. I got 100%. Then I scoped tighter — proximity only, a window right around the number — and got 40%, because 40%, on top of it a 35% recurring puts recurring within sixty characters of the 40.

What finally worked wasn't a smaller window. It was a rule about what lies between:

const KY_HAN = /recurring|for life|lifetime|every month|month after month/i;
const sauDo = text.slice(m.index + m[0].length, m.index + m[0].length + 60);

// If another % sits between this number and the term keyword,
// the keyword belongs to THAT number, not this one.
const dinhSau = KY_HAN.test(sauDo)
  && sauDo.search(KY_HAN) < (sauDo.indexOf('%') + 1 || Infinity);

An intervening percentage sign is the signal. Not distance — interruption. If another number stands between this number and the word recurring, then recurring is describing the other one.

purevpn: 100% → another % before any keyword → rejected. 40% → another % before the keyword → rejected. 35% → keyword first → accepted. 35% lifetime. Correct.

And then I wrote the tests I should have written first

Nine bugs in, I finally did the obvious thing: pinned all ten known cases into a test file.

✓ purevpn (real page text)    ✓ circle
✓ systeme                     ✓ scalenut
✓ kartra                      ✓ snov
✓ lemlist                     ✓ hunter
✓ swipepages                  ✓ fliki
All 10 cases pass

It paid for itself on the first run. The proximity fix I was about to ship broke hunter.io — Earn $27.93 per referral every month. Refer 10 people and get $297.30 is two sentences, and my aggregate-rejection guard (refer \d+) was reading the second one and throwing away the correct number in the first. Exactly bug #9 again, in a different extractor, and I would have shipped it.

Nine bugs found by reading output. One found by a test, three minutes after the test existed.

Notice the shape of this: fix #6 caused bug #9. Narrowing the scan from the page to a window fixed one class of error and created another. The window was the wrong abstraction — close enough to look right for 150 programs, wrong enough to invent terms for one of them.

What hand-checking actually found

I took the top four lifetime programs and read their pages myself. Not a sample — the top four, because those are the ones anyone would act on first.

Vendor	Parser said	Page said
purevpn.com	100% lifetime	35% lifetime	✗ wrong
systeme.io	60% lifetime	60% lifetime	✓
scalenut.com	50% lifetime	"30%–50%" — range	✗ overstated
circle.so	30% lifetime	30% lifetime	✓

Two of four wrong at the top of the ranking. One invented, one quietly taking the best end of a range nobody reaches.

Both are now fixed, and the ranges resolve to their lower bound. But I want to be precise about what that does and does not mean: I fixed the two errors I found by reading four pages. There are 151 programs in the table.

What the data actually says

After all six fixes, cross-checked against five programs I'd read by hand:

Payout structure, n=151

Type	Count
Doesn't disclose	58
Recurring, unspecified duration	43
Fixed term (mostly 12 months)	30
Lifetime	17 19
One-time	3

The 17 lifetime programs, by headline rate: systeme.io 60% (permanent cookie), moosend 40%, purevpn 35%, then a cluster at 30% — thinkific, vidyo.ai, swipepages, mailerlite, circle.so, fliki, koala.sh, scalenut — down to kinsta at 10%.

Three things stand out.

"Recurring" usually means twelve months. It reads like forever. Of the 30 fixed-term programs nearly all cap at 12, and several bury that in an FAQ while the hero section says recurring unqualified. Kartra is the clearest case: the headline says "recurring", the FAQ says "We do not offer lifetime commissions."

38% disclose nothing. 58 of 151 state a percentage and stop. Not fine print — absent. Whether it recurs, for how long, what the cookie window is: you find out after you sign up.

The headline rate is the least reliable field on the page. It's in the largest type and it's the one most often qualified into something else by a clause elsewhere — up to, first 12 months, tier 3, first payment only, or a range whose top end almost nobody reaches.

Networks: PartnerStack 37, Impact 23, FirstPromoter 11, Awin 5, Tapfiliate 4, then a long tail. PartnerStack's dominance in SaaS is real and it is not close.

What I couldn't reach

281 of 554 returned 404 on every probed path. Either no public program, or it's gated behind a form.
61 returned 403 — Cloudflare bot protection. Recorded as 403 and left alone. The tool doesn't rotate user agents, solve challenges, or pretend to be a browser. If a site says no, that's a data point, not an obstacle.
19 had a page but nothing extractable.
The terms that matter most aren't on the page anyway. Which countries can withdraw, tax handling, approval time, clawback windows — mostly visible only from inside the dashboard.

The crawler also killed itself

Worth mentioning, since it cost me a full run. At domain 454 of 554, the process died:

SocketError: other side closed
    at TLSSocket.onHttp2SocketEnd (node:internal/deps/undici/undici:8758:26)

A server half-closed an HTTP/2 connection and undici emitted error on an internal stream — outside the promise chain. try/catch around await fetch() cannot catch that, and an unhandled error event takes the whole process with it.

process.on('uncaughtException', (e) => {
  loiNgoai++;
  console.error(`  ! out-of-band error #${loiNgoai} (skipped): ${e.code || e.name}`);
});

Crude, and I would not ship it in a service that writes anything. In a read-only crawler it's the right trade. The final run swallowed 44 of these — 44 chances to lose twenty minutes of work.

Everything survived only because the database is written per-domain as the scan goes, not at the end. That was luck, not foresight.

The thing I keep coming back to

Every one of the nine bugs is the same bug.

The page says one thing. The machine reads another. Not because the regex was weak — because these pages are written to sell, not to be read. The negation that reverses the cookie. The adjective that outranks the number. The calculator that fills the DOM with plausible decoys. The 100% that turns out to describe how punctually they pay. The $23,940 that exists only inside a marketing screenshot.

None of that is adversarial. Nobody at lemlist built an earnings slider to break my scraper. It's just that persuasive writing and machine-readable writing pull in opposite directions, and every page I hit was optimized for the first one.

So the rule I ended with: the crawler narrows 554 down to 151. It does not tell you the truth. Before any of these numbers goes anywhere that matters, open the vendor's page and read it yourself. I fixed six of these before I trusted the output. Then a wider scan handed me two more, reading four pages by hand handed me a ninth, and the test suite I should have written on day one caught a tenth within minutes of existing.

Nine of those ten were found by a human reading output and thinking that can't be right. If you take one thing from this post, take that: a scraper's failure mode is not crashing. It's returning a number.

707 lines across four files, Node standard library only. No affiliate links in this post — I'm not enrolled in any of these programs, and the rankings are mechanical output, not endorsements. Numbers verified against live pages on the day of publication; vendors change terms without notice.

DEV Community: Tien Luong