A "monthly" app plan cost me $300 to leave. Now a local Gemma reads the terms before I click "Start free trial"

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady

What I Built

I signed up for an app's free trial, and it turned into a monthly plan. A few weeks later I tried to cancel and found out I couldn't for three months, even though it was billed monthly. Leaving early cost $300. The rule was in the terms I had scrolled past on the way to the "Start free trial" button.

I'm not going to start reading terms of service. So I built Catch, a Chrome extension that reads them for me at that button.

When a sign-up or checkout page has a price and a button, Catch reads the text around the button and the terms page it links to. Gemma 4 E2B, running in Chrome on my own GPU, double-checks the sentences that look like trouble, and Catch pins a card next to the button:

• Exit fee: "Leaving early costs $300."
• Lock-in: "You're committed for 3 months, even though it's billed monthly."
• Trial turns paid: "The trial turns into a paid plan at $29.99."

Every line quotes the sentence it came from. Click it and the page scrolls to that sentence, or the terms page opens with it highlighted.

If I press the button anyway, Catch writes my exit plan: the last day to cancel for free, the day I can leave without the fee, and how cancelling has to be done, as events in my calendar. Then I can stop thinking about it.

It works on real terms too. On Xfinity's subscriber agreement it pulls out "If your Services are subject to a minimum term agreement, and you terminate… before the end of the minimum term, you may be charged an early termination fee." That's the kind of sentence I would never have scrolled to.

Demo

Try it without installing anything: https://kinnuworks.github.io/catch/#try

It's a made-up sign-up page with the same trap I fell for, and the extension's own code runs inside the page. Hover the button, read the card, press "Start free trial" for the exit plan. The demo starts with the rules, which work instantly; switch to Gemma 4 E2B to watch the model double-check them (a 3.4 GB download, once). There's also a box where you can paste the terms from any real sign-up page.

Code

kinnuworks / catch

A Chrome extension where a local Gemma reads the terms at the sign-up button and writes your exit plan.

Catch

Catch is a Chrome extension that reads the terms on a sign-up page at the moment you're about to press "Start free trial". It pins a card next to the button with the catches that cost money: renewals, lock-ins, exit fees, cancellation rules and refunds. Each line quotes the sentence it came from. If you press the button anyway, Catch writes your exit plan, with the last day to cancel for free, as calendar events.

Gemma 4 E2B runs in your browser on your own GPU. The page text never leaves your laptop.

I built it after an app's free trial turned into a "monthly" plan I couldn't cancel for three months. Leaving early cost $300. The rule was in the terms I scrolled past.

Live demo, no install: https://kinnuworks.github.io/catch/

Video (70 s): https://youtu.be/3MkDbNKn8Ng

How it works

sign-up page ──► find the button ──► read the text around it

…

View on GitHub

The repo has the extension, the landing page, 32 tests, and the whole evaluation: the scripts that fetched the terms, the reviewed labels, and every reader's raw output, including each probability Gemma gave.

How I Built It

Code reads, Gemma checks

The obvious version asks a model to read the terms and say what's wrong with them. I trust that version least. A model that writes the warning can also write a warning that isn't in the document.

So Catch splits the job:

1. Code finds sentences. A splitter that doesn't break on "$29.99", "e.g." or "section 4.2" cuts the terms into sentences and remembers where each one sits on the page.
2. Rules cast a wide net. Fifteen patterns pick sentences that might be about renewals, trials, minimum terms, exit fees, cancellation rules or refunds, and guess which kind.
3. Gemma answers yes or no. For each guess it gets one narrow question, such as "Does this sentence say the customer must pay a fee, or the remaining payments, if they cancel early?"
4. Code writes the card. The line on the card is built from amounts and durations that code pulled out of the quoted sentence. Gemma can't turn $300 into $30, and it can't add a clause that isn't on the page.

The rules alone are fast and catch real things, but they cry wolf. On Peloton's terms they flag "30 days notice, in writing", which turns out to be about rejecting changes to the arbitration clause. On 1Password's, "within thirty (30) calendar days… in writing" is about erasing data after you leave. Gemma 4 E2B gave those two a 0.44 and a 0.29 chance of being a real cancellation rule, so they stay off the card.

Picking the right Gemma size

I started with Gemma 3 1B because it's an 800 MB download. I asked it eight questions where I knew the answer and looked at how likely it thought "yes" was:

Sentence	Should be	Gemma 3 1B	Gemma 4 E2B
"All amounts paid are non-refundable."	yes	1.00	0.96
"You may cancel at any time and receive a prorated refund…"	no	1.00	0.06
"…Spotify shall have no liability to you." (asked: lock-in?)	no	1.00	0.07
"…terminate before the end of the term, you will be charged an early termination fee."	yes	1.00	0.93
"We will provide at least 30 days' notice prior to any new terms…"	no	1.00	0.20
"Subscriptions automatically renew until cancelled…"	yes	1.00	0.95
"Credits are non-refundable, non-transferable…" (asked: lock-in?)	no	0.97	0.01
"…Babbel will provide you with a pro rata refund…" (asked: exit fee?)	no	1.00	0.24

Out of the box, at 4-bit, with my zero-shot prompt, the 1B said yes to almost everything, so on top of the rules it added nothing. Adding worked "no" examples to the prompt didn't change it. Gemma 4 E2B separated the cases without any fine-tuning, so that's what ships. A 1B fine-tuned on these labels might well do this job; that's the next experiment.

Trusting the confidence, not the word

Even with the bigger model, I don't take the single word it writes. Catch runs one forward pass, looks at the model's scores for "yes" and "no" as the next word, and turns them into a probability. That gives a number between 0 and 1 for every question, so the cut-off is a choice I can measure instead of a guess. Tuned for the best balance on half of my labelled services and tested on the other half, it lands between 0.05 and 0.23. For the card I use a stricter 0.5: it shows fewer catches, but most of what it shows is real, and a card that cries wolf teaches you to ignore it.

Dates are arithmetic, not AI

The exit plan never asks the model for a date. Code finds "7-day free trial", "three (3) months" and "at least 14 days before your next billing date", even when they sit in different sentences, and does calendar maths: three months from January 31 is April 30, not "90 days later". The calendar file opens in Apple Calendar, Google Calendar and Outlook, with a reminder two days before each deadline.

27 real terms documents, labelled by hand

I fetched the terms of 50 subscription services; 27 could be read without running their JavaScript. Catch picked up to 8 candidate sentences from each, 136 in total. An AI assistant (not Gemma, and not the rules) suggested a label for each, and I went through all 136 one by one in a review page before any reader's output was scored. I didn't change any of them, which makes the labels only as good as my review, so every file is in the repo to check. The labels, the suggestions and the review page are all in the repo.

Reader	Precision	Recall	F1	Time per document
Rules only	63%	91%	75%	under 1 ms
Gemma 3 1B, threshold tuned per fold	64%	91%	75%	0.6 s
Gemma 4 E2B, asked about every kind (0.5)	90%	65%	76%	1.7 s
Gemma 4 E2B, rules decide lock-ins (0.5, what ships)	85%	72%	78%	1.4 s

Precision: of the catches shown, how many were real. Recall: of the real catches among these sentences, how many were shown.

The rules alone flag almost every real catch, but more than a third of what they show is wrong. With Gemma 4 E2B checking them, false alarms drop from 37% of the card to 15%, and the card keeps nearly three quarters of the real catches. The 1B made no difference at all, which matches the probe table above.

The first Gemma 4 run taught me where it's weak: it rejected all nine real lock-in sentences, such as "payment obligations are non-cancelable" in Zoom's terms. It reads "minimum term" language as neutral. So lock-ins now skip the model and come straight from the rules, which is how the demo card can still show my three-month lock-in, and fine-tuning for lock-ins is the next job. Gemma also accepted Babbel's fee for skipping a booked class as an exit fee (0.92), a fair reading of the words but not what Catch is for.

Problems I hit

• The model wouldn't stay cached. Gemma 4 E2B is two weight files of 1.5 and 1.6 GB. In one browser profile with a 3.9 GB storage quota, the browser silently refused to cache them, so every reload downloaded them again. The extension now asks for unlimitedStorage.
• Chrome kept unloading the model. Extension service workers are stopped when idle, which dropped the model from the GPU every few minutes. It now lives in an offscreen document.
• "twenty-four months" meant four. A reviewer found that my number parser skipped "twenty" and read "four". Compound numbers, European prices like "29,99 €", and a daylight-saving bug that moved a deadline by a day all have regression tests now.
• The exit plan vanished on real checkouts. The first version attached its click listener after the model finished, and a real "Start free trial" navigates away. Now it listens from the start, saves the plan the moment you press, and shows it again on the next page.

Where it's still wrong

• It only checks what the rules flag. All 136 test sentences were picked by the rules, so the recall numbers above mean "recall among sentences the rules found". A trap written in words my fifteen patterns don't expect is invisible to Catch and to this test.
• The test runs the same weights in a different runtime. The numbers come from Gemma 4 E2B at 4-bit through Ollama on my laptop; the extension runs the 4-bit ONNX build of the same model in the browser. Small differences between the two are possible.
• Exit fees were rare in my sample. Only two of the 136 sentences were real exit fees, so that row says little on its own. The exit-fee trap I fell for is in the demo, not in the test set.
• English only, text only. Terms inside images, PDFs or a payment provider's iframe are skipped.
• It's not legal advice. It points at the sentences worth reading yourself.

Why Does Open Innovation Matter?

It reads every sign-up page I visit. A version of Catch that sent page text to a hosted API would be a log of everything I almost bought. With open weights running in the browser through WebGPU, the page text never leaves the laptop. The only requests are the one-time model download and the site's own terms page, fetched without cookies.

I could take the model apart. The yes-bias only showed up because I could read Gemma's next-token scores directly, and calibrating a threshold on them is what made the 2B model usable. With the weights and the runtime in my hands, I can measure, swap models, or fine-tune.

It costs nothing to run. There's no per-page bill, so there's no reason for Catch to have a subscription.

Where a closed model would be better: coverage. A large hosted model reading the whole document would probably catch traps written in words my fifteen rules never look for. Catch only checks what the rules flag, and I'd rather miss a strangely worded clause than send my browsing to a server.

Prize Categories

• Best Use of Gemma: Gemma 4 E2B runs in Chrome through Transformers.js and WebGPU. Catch reads its yes/no probabilities from the logits and calibrates the threshold on hand-labelled data.
• Best Use of ElevenLabs: the demo video's narration is my own voice, cloned with ElevenLabs and generated line by line from the script.

I built this with help from an AI coding assistant. Every number in this post comes from the evaluation files in the repo.