DEV Community

Vanshaj Poonia
Vanshaj Poonia

Posted on AI-assisted

The contract you can't paste into ChatGPT: a red-flag finder that runs in your browser

Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle Submission ๐Ÿ’ธ

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

Every freelancer has signed a contract that the client's lawyer wrote. Twelve pages, and somewhere in clause 6.1 you have agreed not to work for anyone in "the beverage industry" for eighteen months, in 4.1 your logo sketches became their property the moment you drew them, and in 2.2 and 2.3 you get paid ninety days after they decide they are happy.

Nobody reads that carefully at 11 pm before a kickoff call. A lawyer would, at a few hundred dollars an hour. A chatbot would too, except that a lot of client contracts are confidential, and pasting one into a hosted AI service means handing it to a third party.

Fine Print reads the contract for you, on your own device. Paste it or drop in the PDF, and it marks the clauses freelancers regret:

  • red flags: you hand over ownership of your work, non-compete, exclusivity, no limit on what you could owe, you give up the right to challenge them, unlimited revisions;
  • things to negotiate: they can cancel at any time, fixed penalties, no-poach clauses, auto-renewal, non-disparagement, slow payment, "paid only if they are satisfied";
  • good to know: insurance you must carry, audit rights, the liability cap, whose law applies.

Each one shows the clause, what it means in plain English, and what to ask for instead. Then Draft my reply turns the ones you tick into a polite email to the client, with section numbers and blanks for your own terms.

Fine Print's report on a freelance design contract: four red flags, each with the clause, what it means and what to ask for

Who pays for this? Freelancers who sign a handful of client contracts a year and cannot justify a lawyer for each one: designers, developers, writers, consultants. And, I think, the groups they belong to. A freelancer collective or a coworking space that wants to offer members a contract check has a problem with every hosted option: it would become the custodian of its members' clients' contracts. Fine Print never receives a contract, so there is nothing to custody. That is also what makes it a viable one-person side business: no servers to run, nothing to leak, and no cost per scan, so it can be priced like a tool (a few dollars a month, or a one-off fee) instead of like a lawyer.

Demo

๐Ÿ”— vanshajpoonia.github.io/fine-print

No sign-up. There are two samples:

  1. A freelance design contract I wrote for the demo, with the usual traps planted in it. Fine Print flags 15 things, and Draft my reply writes this:

The reply Fine Print drafts from the ticked flags: numbered points with section numbers and blanks for the freelancer's own terms

  1. A real contract, with the lawyers' answers: one of the 102 contracts the model never saw in training, with what the lawyers marked shown next to what Fine Print found. It is an honest sample. The lawyers marked 7 kinds of clause; Fine Print finds 4 and misses 3. I left it in rather than hunting for a contract that flatters it.

On a real held-out contract: Fine Print found 4 of the 7 kinds of clause the lawyers marked, and each card says whether the lawyers marked it too

The first scan downloads EmbeddingGemma (197 MB) once. After that, on my laptop, the sample contract (41 clauses) is read in 0.7 seconds with WebGPU, or 12 seconds on a browser that only has the CPU, and it all works offline.

Code

GitHub logo VanshajPoonia / fine-print

Finds the clauses in a freelance contract that cost you money. EmbeddingGemma reads every clause in your browser; nothing is uploaded.

Fine Print ๐Ÿ”

Finds the clauses in a client contract that cost freelancers money, before you sign it.

Paste the contract or drop in the PDF. EmbeddingGemma, Google's open-weight embedding model, reads every clause in your browser, and a small classifier trained on 350 contracts that lawyers had marked up flags the ones to worry about: non-competes, IP grabs, no cap on what you could owe, penalties, auto-renewal, cancel-at-any-time and more. Each flag comes with what it means and what to ask for instead, and Draft my reply turns the ones you tick into an email to the client.

Live: https://vanshajpoonia.github.io/fine-print/

Fine Print's report on a freelance design contract: red flags first, each with the clause, what it means and what to ask for

Why it runs on your device

Client contracts are confidential, often explicitly. Many say so in their own confidentiality clause, and pasting one into a hosted AI service means handing it to a third party. Fine Print never has the contract: the model is a file your browserโ€ฆ

Static files, no build step, no backend, no API keys. MIT.

How I Built It

How Fine Print reads a contract: clauses go through EmbeddingGemma and 14 small classifiers in the browser; everything shown about a flagged clause was written by a person

The model finds. People wrote the advice.

The first decision was what not to let an AI do. A small model explaining a legal clause in its own words is exactly where you do not want invention, so nothing a freelancer reads in Fine Print is generated. Every explanation, every "what to ask for" and every line of the reply email was written by hand, once, per kind of clause.

The AI's job is the part that can be measured: deciding which clauses to show you. That is a classification problem, and the pipeline is small:

  1. Split the contract into clauses.
  2. Turn each clause into 768 numbers with EmbeddingGemma, Google's 300M open-weight embedding model, running in the browser through transformers.js (4-bit, WebGPU, falling back to WASM).
  3. Run those numbers through 14 tiny classifiers, one logistic regression per kind of clause. All 14 together are a 79 KB JSON file.

One snag worth knowing if you ship EmbeddingGemma to browsers: the 4-bit file uses a quantised token-lookup operator that WebGPU has and the WASM runtime does not, so it fails outright on any browser without WebGPU. My tests passed, because Node's runtime has the operator; a test run with the GPU switched off caught it. Browsers without WebGPU now load a sibling build that keeps the lookup table unquantised, and I measured the difference instead of assuming it: on 2,936 test clauses, 36 of 41,104 flag decisions differ (0.09%).

Three pattern checks cover freelancer problems no dataset labels: payment more than 30 days after invoice, acceptance "to the Company's sole satisfaction", and unlimited revisions. The app marks those as "found by a rule" so you know which is which.

The data: 510 contracts that lawyers already read

The classifiers learn from CUAD, the Contract Understanding Atticus Dataset: 510 real commercial contracts from SEC filings, with clauses in 41 categories highlighted by lawyers and law students, released under CC BY 4.0. Fine Print's 14 flags map onto 16 of those categories.

I used CUAD's official split and never touched its 102 test contracts until the end. The 408 training contracts were split again: 350 to fit the classifiers, 58 to choose each one's threshold.

One unglamorous problem first: CUAD stores every contract as a single line with all whitespace collapsed. A real PDF has hard-wrapped lines instead. The segmenter has to handle both, splitting on sentences and numbered clauses while not splitting "Inc.", "Section 9.2" or "U.S.", and keeping short headings like "Non-Competition." attached to the clause they introduce, because a heading is often the best clue there is.

Measuring it, three ways

npm test runs the real model over all 102 held-out contracts: 20,671 clauses, 824 of them highlighted by lawyers under one of the 14 flags. Every approach is scored the same way.

Approach Clauses found Flags that were right False alarms per contract Contracts: found / right
Ctrl+F (hand-written keyword patterns) 49% 29% 10.5 71% / 70%
Zero-shot (similarity to a one-line description) 38% 7% 44.3 69% / 56%
Trained classifiers on EmbeddingGemma (shipped) 66% 56% 4.6 83% / 78%
Same training on bge-small (a sixth of the size) 60% 48% 5.5 81% / 74%

The last column is the question a freelancer actually asks: does this contract have a non-compete? When the lawyers found one, how often Fine Print says so; and when Fine Print says so, how often it is right.

Zero-shot was my first idea, and it is useless here. Describe a non-compete in a sentence, flag clauses whose embeddings are close to it: it finds some, and buries them under 44 false alarms per contract. To an embedding model, all legalese looks like all other legalese.

Ctrl+F is a better baseline than I expected. "governed by" finds governing-law clauses perfectly well. It falls apart on anything that can be said more than one way. A covenant not to sue is "shall not challenge", "shall not contest", "will not directly or indirectly attack".

Learning from the lawyers is what works. Training a classifier per flag on the lawyers' highlights found more clauses than either baseline, with a fraction of the false alarms. It is strongest on governing law (96% of clauses found), uncapped liability (94%) and insurance (90%), and found 70% of non-compete clauses, flagging 87% of the contracts that have one.

The 300M model earned its 197 MB. I trained the same classifiers on bge-small, a sixth of the size. Overall it was not far behind (60% of clauses found against 66%), but on non-competes, the clause a freelancer most needs to see, it found 44% against EmbeddingGemma's 70%.

It is weakest on penalties (22%) and IP ownership (36%), partly because CUAD's categories are broader than a freelancer's worry. Its "IP ownership assignment" includes things like two companies handing domain names back and forth, which nobody needs flagged in a design contract.

Not every false alarm is false

Reading the "false alarms" to choose quotes for this post, I kept finding clauses like this one, flagged as a liability cap and counted as wrong:

THE TOTAL INDEMNITY SHALL NOT EXCEED IN ANY CASE THE AMOUNT INSURED SET FORTH FOR SAID PURPOSE.

That is a liability cap. CUAD's annotators often highlight one instance of a clause per contract, so a second one goes unmarked. So I took 40 false alarms at random (node scripts/audit.mjs 40 prints the same 40): 8 are plainly the right kind of clause, 12 sit right next to one (its heading, a carve-out, a recital), and 20 are plain mistakes, like a quote from a press release flagged as exclusivity. That is my reading, not a lawyer's, and every number in the table still counts all 40 as wrong.

What it cannot do

It cannot read scanned PDFs (pictures of text). It is English-only, trained on US commercial contracts. It points at clauses; it does not know your situation, and it can miss things. The footer says so: not legal advice. It just helps to know what to ask the lawyer, or the client.

Why Does Open Innovation Matter?

Fine Print only works as a product because the model is open. Four reasons:

The document is the whole problem. A contract worth checking is usually one you are not supposed to share. With EmbeddingGemma running in the tab, "your contract never leaves your device" is not a privacy policy that can be revised later. It is where the code runs. Open the network tab, scan a contract, and watch nothing go out.

Top comments (0)