Fineprint: Find out what really changed in the ToS

This is a submission for the Postmark Challenge: Inbox Innovators.

What I Built

I built Fineprint, a service for figuring out how a Terms of Service, Privacy Policy, User Agreement, or other company legal document has changed.

The idea is that companies are always sending "Upcoming changes to our legal agreements" emails, but often don't tell you what has changed.

That's where Fineprint comes in!

Just forward one of those "Our Terms of Service has changed" emails to [e-mail]. It'll load the policy, find a previous version (from the Internet Archive), calculate a diff, and summarize the changes for you.

Why I Built Fineprint

[note: you can skip on down to the Demo section below if you just want to see it in action]

When I saw the contest idea, I asked myself "what is unique to email?". What properties and what communications make the most sense over email? After all, there's a dozen or so different form factors for communication these days: social media, SMS, phone calls, websites, apps, WhatsApp/Signal/Discord/etc, snail mail, carrier pigeon, smoke signals, etc.

I threw down a few random thoughts:

• Forwarding stuff is a quintessentially "email" thing to do
• Email is a common mechanism for receiving notifications

And from there, the question was "what notifications are specific to email?". There are many notifications that we receive through email that are redundant -- updates from LinkedIn, GitHub, Discord, social media, etc -- but those all have some other source of truth we can query directly. What notifications would we not receive anywhere else?

And the answer, at least for me, was policy updates.

Policy updates are perfect because it's frequently a legal requirement for companies to notify you when they change, and the main medium for doing that is email.

While a select few companies will go the extra mile to say what they've changed, the vast majority just say "here's the new policy, go read it or something".

And of course, basically nobody reads the Terms of Service, and similarly few people are going to go to the effort to diff two versions of a 10+ page document. But these things are also important! Is the company now collecting information I would consider invasive? Are they sharing my information with new partners that I think are shady? Are they selling access to the information I've given them?

It turns out computers can make quick work of this task. And that's what Fineprint is for -- lowering the activation energy to answering these questions so you can make informed decisions about which companies you interact with and entrust your information to.

Demo

The best demo is just to try it! Search your inbox for subjects like "Changes to [...]" or "[...] Agreement Update" or "Update to [...]", and forward one of those emails to [e-mail].

The response you'll get will look something like:

In this case, we couldn't find an older version of the policy because the Internet Archive is down this morning 🙃

And an example with functional diffing, now that Internet Archive is back up:

More details are available at fineprint.help. There are heavy rate limits in place as a cost control, as I'm using several APIs to generate the policy summaries.

Instructions for running the application locally are included in the GitHub repo.

Code Repository

bcspragu / fineprint

A tool for determining what has changed in a company's legal agreement (ToS, Privacy Policy, etc), built for Postmark's Inbox Innovators challenge

Fineprint

A tool for determining what has changed in a company's legal agreement (ToS, Privacy Policy, etc), built for Postmark's Inbox Innovators challenge

More usage details at Fineprint.help

Local Testing

Dependencies

To run the service locally, you'll need all of the following:

• A Go compiler (likely 1.24.x)
• A Node environment (I'm using Node 22)
  • mjml needs to be installed in this env, for formatting emails
• Postmark credentials
• Internet Archive keys
• An Anthropic API key

Running

./scripts/run.sh is a helper for running the service, but it assumes you have your credentials stored in pass under specific names.

curl -v \
  -u $(pass show postmark/webhook-username):$(pass show postmark/webhook-password) \
  --data @json-body.json \
  -H 'X-Forwarded-For: 127.0.0.1' \
  localhost:8080/webhook

Limitations

• If the legal document has changed a lot, the diff may be very large and overflow our LLM context, so we trim it down to size
  • This…

View on GitHub

For testing locally, follow the instructions in the README.md file in the root of the repo. It will require a functioning Go environment, as well as an Anthropic API key, Postmark credentials, and Internet Archive keys. You'll also need a Node environment with mjml installed.

How I Built It

The first ~50-60% of this project were written entirely with Claude Code, and then I came in and cleaned up the API calls, added the diffing support, tuned the prompts, added the MJML templates, and generally built out the rest of the project.

The trickiest part was handling the different possible cases: e.g. ToS;DR results available or not, older versions of the policy available or not, etc. There are also lots of heuristics required to get the correct URLs + policies and correlate them across services.

The server that receives the Postmark Inbound Email Processing webhook is written in Go. It uses Claude for classifying the email and generating summaries. The ToS;DR API is used to get curated write-ups of policy information and other metadata. Postmark is used for both the inbound (i.e. forwarded policy) emails, as well as for sending the reply emails, which are sent via the API.

Working with Postmark

Working with Postmark was very straightforward: the APIs are clearly documented, use a standard HTTP/JSON interface, and are well-structured. The console itself is easy to navigate to help with testing and debugging, and adding basic auth support to the webhook was a breeze.

Comments

[Dotallio]

Super helpful! Turning those 'we updated our terms' emails into actual summaries solves a problem I've hit so many times.

Do you think you'll expand Fineprint to cover things like changelogs or software licenses too?

[Business Attire]

Thanks! As for expanding to other areas: Changelogs are nice because they usually already include the specific delta between versions. Software licenses would be an interesting one, but one doesn't usually receive email notifications when they change.

I think one could definitely expand the existing infrastructure to support more generic diffing (and even proactively watching) from other sources, but the 'send an email' form factor is especially conducive to policy updates (because it's frequently a legal requirement to send an notification when you change your policies). Other types of version-diffing summaries might work better in another form factor, like a website or app.

[Nikoloz Turazashvili (@axrisi)]

very cool! Does it also somehow handles the multiple previous versions? so it knows what to compare to the new one?

[Business Attire]

Thanks for the comment! The heuristic it uses is to find the latest version that exists up until a week before the date of the email.

The reason is that if you grab too recent of a snapshot, there might be no diff, as you're just grabbing the current version.

A smarter approach beyond this MVP would be to iteratively look backwards at recent snapshots until you find the diff.

As for long-running historical change tracking, that's probably better suited to a website than an email-based flow.