Reads everything. Remembers nothing: an email watch API built on open-weights AI

Seth S — Tue, 13 Oct 2026 01:30:46 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

WatchFox is an email-watching API. You tell it which mailbox to watch and what to catch (an enquiry, an invoice, a support ticket, a purchase order, a job application, a claim, or a type you define), and when a matching email arrives it calls your server with a signed webhook. Your app never has to read an inbox or write a classifier.

Who pays: SaaS builders and operations teams that need to react to inbound email: sales enquiries and RFQs that must reach the right person in minutes, invoices heading into accounts payable, support requests, purchase orders, claims. Their customers connect a mailbox through a hosted page that carries the SaaS's own name, logo and colour, so the SaaS can offer "connect your mailbox" in an afternoon.

The problem it solves: every team that wants this ends up writing the same fragile pile: IMAP polling, regex rules, an LLM prompt per email, and a privacy question nobody can answer ("where did our customers' emails go?"). WatchFox replaces that with one API call, and answers the privacy question in the strongest way possible: it never stores email content.

It's live and taking subscriptions: Kit (free, 50 runs a month), Tracker ($29) and Alpha ($149).

Demo

Live site: https://watchfox.run
Docs written for humans and for coding agents: https://watchfox.run/#/docs, plus https://watchfox.run/llms-full.txt (the whole API in one plain-text file an agent can read)

Code

sethsazure / watchfox-open

WatchFox: an email-watching API. Signed webhooks, open-weights decision model (Laya typed-decisions), zero email retention.

WatchFox

An email-watching API. Tell it which mailbox to watch and what to catch (an enquiry, an invoice, a support ticket, a purchase order a job application, a claim, or a type you define). When a matching email arrives, WatchFox calls your server with a signed webhook Your app never has to read an inbox or train a classifier.

Live service: https://watchfox.run (free plan, 50 runs a month; the "try it" box on the home page runs the real checker without signing up).

Open-weights decision model, self-hosted. Emails are judged by convaiinnovations/laya-typed-decisions a small (421M parameter) encoder that answers typed, calibrated questions, served by the open-source laya-serve. It is not a chat model: no tokens, no per-email API bill, no invented answers, and a score you can put a threshold on. On a single 12 GB consumer GPU it makes roughly 400 decisions per second when requests…

View on GitHub

About 6,400 lines of Python (FastAPI), 3,500 lines of React, 74 automated tests that run on both SQLite and PostgreSQL, and a GitHub Actions pipeline that tests, builds and deploys to Azure on every push.

How I Built It

The open-source AI: an open-weights "decision" model, self-hosted.
WatchFox doesn't ask a chat LLM "is this an invoice?" on every email. It uses Laya typed-decisions (open weights on Hugging Face), a 421M-parameter ModernBERT-large encoder that answers typed, calibrated questions about a piece of text: yes/no probabilities, choices and ordinal scores. It runs on laya-serve (open source) on In my own server rack, and the production app reaches it over a private network.

Why that choice, in numbers: one non-generative forward pass, about 3 GB of VRAM, and **roughly 400 decisions per second when requests are batched. No tokens, no per-email API bill, no hallucinated answers, and a score I can put a threshold on.

Problems I had to solve (the interesting part):

A model that wasn't built for my job. The base model answers questions; it doesn't know what a "good" question is for my customers. I built a tuner: for each type you create, it tries several wordings of the question, scores them on your labelled examples and on a built-in set of look-alikes, picks the wording that separates them best, sets the threshold, and adds exclusions for the look-alikes it still gets wrong. Press "wrong catch" on a real email and it re-tunes from that mistake. It's measured, not magic, and the UI says so.
"Never keep the email" is easy to promise and hard to build. WatchFox keeps only the subject, sender, a 300-character snippet, attachment names and scores, for 90 days. When a customer needs the whole .eml or an attachment, WatchFox reads it from the mailbox at that moment, streams it through, and forgets it. The full text only ever exists in memory while a webhook is being delivered.
Putting a private GPU behind a public web app. Azure App Service containers have no network device that talent recognizes, so the node never signed in. Fix: userspace networking plus the TAILNET flag, and a SOCKS proxy that carries only the model calls, so nothing a user supplies (a webhook URL, a mail-server host) can ever reach the private network. If the GPU is unreachable, a paced fallback queue takes over, and the status page says which path is serving.
Every limit is the same in the dashboard and the API. A test hits each limit with an API key, because that's where limits usually leak.
Built for AI agents as customers. llms.txt, llms-full.txt and an OpenAPI file are generated from the live API, and a test fails if anyone adds a public endpoint without a description. A coding agent can integrate WatchFox from one file.
Proven under load. On a local PostgreSQL run with the model stubbed: 100 accounts, 5,000 emails, every webhook delivered in about 31 seconds with zero errors, while 50 clients hammered the read API (median 0.3 s).

Stack: FastAPI, React + Vite, PostgreSQL, Azure App Service (containers), GitHub Actions with Azure OIDC (no stored cloud password).

Why Does Open Innovation Matter?

Three things a closed API would have made impossible:

The privacy promise. "We never keep your email" means nothing if the text goes to someone else's LLM first. With open weights on my own GPU, an email's text goes from the mailbox to my model over a private network and nowhere else. I can tell customers exactly where it went.
The economics of a side hustle. A per-token API bill scales with every email. A 421M-parameter open model on a GPU costs the same at 10 emails or 10 million.
Stability. While preparing this launch I found that a closed model my tooling relied on was blocked for my Azure subscription in my region, and its successor wasn't offered there. I had to change the assistant's model mid-build. The open weights I run don't get retired out from under me: I choose when to change them, and I can re-tune and re-test on my own data.

My Agent Session

I built WatchFox in a long pairing session with Claude Code: design, the tuner, the security audit, the load test, the Azure and GitHub pipeline, and even the playbook I'm now using to migrate my next app the same way.

DEV Community: Seth S