DEV Community

Param Jaisinghani
Param Jaisinghani

Posted on

My paid AI feature was simultaneously free-for-anyone-to-abuse AND broken for the people actually paying for it

A few weeks ago I shipped "Deep Analysis" and "Advanced Analysis" on APEX Versus - an AI "who would win" comparison site. Pay a few credits, get a deeper AI breakdown of any matchup: per-dimension reasoning, a hypothetical scenario, wildcard factors, that kind of thing. Pro plan users are supposed to get it unlimited.

Ran a full debug pass on the codebase this week. Found out the feature had two bugs at once, and they canceled each other out in the worst possible way.

Bug 1: the paywall didn't exist server-side.

The frontend checked credits before calling the API. Normal enough - except the API itself never checked anything. POST /api/compare with mode: 'advanced' and no auth header at all would happily burn a real Gemini call and return real content. No login required, no credit check, no daily cap. The free-tier quota I did have only applied to the basic comparison endpoint - I'd built rate limiting for the wrong door.

I confirmed it with a plain curl request. It worked. Zero auth, zero payment, full output.

Bug 2: the feature was invisible to the people who'd actually paid for it.

Separately, the UI had this line:

deepCard.style.display = isPaid ? 'none' : 'block';
Enter fullscreen mode Exit fullscreen mode

Read that again. The card that lets you use Deep Analysis was hidden whenever the user was on a paid plan. My "Advanced Analysis" button - the one Pro users are paying for - had literally no onclick anywhere in the HTML. The function existed. Nothing called it. Ever. It had been dead code since the day I wrote it.

So: free users who found the right request shape got it for nothing, forever. Paying users couldn't find the button to use the feature they were paying for. Directly opposite of what a paywall is supposed to do, both directions, at once.

The root cause was the same bug wearing two different hats.

I'd tightened Firestore security rules a while back so credit balances can only be written server-side (admin SDK), not by the client - good instinct, closes a real self-serve-credits exploit. But I never went back and updated the code that used to write credits from the browser. It was still there, still running, just silently failing every time:

window.deductPaidCredit = async function(uid) {
  try {
    await updateDoc(doc(db, 'userCredits', uid), { credits: increment(-1) });
    // this write has been rejected by security rules for weeks.
    // the empty catch below means nobody ever found out.
  } catch(e) {}
};
Enter fullscreen mode Exit fullscreen mode

An empty catch block is a great way to make a permission error indistinguishable from success.

Fix was straightforward once the actual shape of the bug was clear:

  • Moved the credit check and deduction fully server-side, inside a Firestore transaction, before the Gemini call happens - not after.
  • Deleted the dead client-side deduction function entirely.
  • Fixed the inverted visibility check so paid users can actually see the feature.
  • Added the missing button for the half of the feature that never had one.

Then I tested it for real instead of trusting the diff: ran Deep Analysis on a live paid account and watched the credit balance drop by exactly 2, atomically, server-confirmed. First time in this feature's existence that a credit deduction has actually happened.

The lesson that's stuck with me: a client-side check and a server-side check are not the same feature just because they look at the same variable. One is a suggestion. The other is the only one that counts. I had the suggestion and skipped the count for a feature that directly touches revenue, and it took a dedicated audit - not normal usage, not testing, not code review - to surface it, because both failure modes were silent by design (empty catches, hidden UI) rather than loud.

If you're running anything with a credits/paywall system: grep your codebase for every place a balance gets checked, and ask whether that check would survive someone skipping your UI entirely and hitting the API directly. If the answer requires the word "should," it's not actually enforced.

APEX Versus is live at apexversus.com if you want to see the (now-actually-working) Deep Analysis in action - first few comparisons are free, no signup needed.

Top comments (0)