This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady
What I Built
Dinner is over, one person paid, and now that person gets homework. Who had the cod? Was the calamari shared? Is the tip split evenly or by what you ordered? Someone opens a calculator, someone else says "just split it evenly", and the person who had a flan and an espresso quietly pays for someone's steak.
Itemised bill splitting exists, but it usually means typing every line of the receipt into an app, or uploading a photo of it (with the last digits of your card, the time and the place you ate) to somebody's server.
Lazy Susan does it on your phone, with no typing:
- Take a photo of the receipt. Donut, an open-weight document model, reads every line in the browser.
- The receipt checks the model. The lines have to add up to the subtotal, or with tax, service and discounts, to the total. If Donut's read does not add up, a second open model, Florence-2, reads the photo again, and the read that adds up wins.
- Pick a person, tap what they had. Shared dishes get tapped for everyone who shared them.
- Tax, service and tip are shared in proportion, everyone's share is exact to the cent, and one button copies a message for the group chat.
Who gets to be lazier: whoever paid. Nobody types a price, nobody does the maths, and nobody has to trust that an AI read it right, because the receipt itself says whether it did.
Demo
🔗 vanshajpoonia.github.io/lazy-susan
No sign-up. Take a photo of a receipt, or tap one of the four samples (real receipts from the two public datasets I tested on):
- Lunch at a deli: Donut reads it, and it adds up as read.
- Portuguese wine bar: Donut reads the restaurant's address, "Palm Coast, FL 32137", as a dish. It is the only line that stops the sums from working, so the app leaves it out and tells you why.
- Steak and salmon: Donut's read does not add up, so Florence-2 takes a second look, and its read does.
- Thai dinner, Indonesia: rupiah, thousands separators, service and tax, the kind of receipt Donut was trained on.
The first photo downloads Donut (219 MB) once; Florence-2 (266 MB) downloads only if a receipt ever needs a second look. After that it works offline. On my laptop (an M4, in Chrome), a receipt Donut can vouch for takes 8 to 12 seconds, and one that needs the second look 21 to 26. I have not measured a phone; expect slower.
Code
VanshajPoonia
/
lazy-susan
Photograph the receipt, tap who had what, paste the split. Two open models read it in your browser; the receipt's arithmetic checks them.
Static files, no build step, no backend. MIT.
How I Built It
The first model: Donut
Donut (NAVER Clova, MIT) is an OCR-free document model: an image encoder and a small text decoder that writes the receipt straight out as tags, <s_nm>Pad Thai</s_nm><s_price>10.95</s_price>, with no separate text-recognition step. I used the version fine-tuned on CORD, 800 Indonesian receipts, quantised to 8 bits, running in the browser through transformers.js.
Indonesian receipts are not American receipts, so the first thing I did was point it at receipts it had never seen: WildReceipt, a public dataset of receipts photographed in the wild, mostly in the US. It reads about half the lines, and it makes one mistake again and again: it thinks the restaurant is something you ordered. "Welcome to Chick-fil-A", at a price of "03270)". "Koho Restaurant", at a price of "Airport". "Palm Coast, FL 32137".
A receipt is a checksum
You cannot ask everyone at the table to proofread an AI. But a receipt carries its own check: the lines have to add up to the printed subtotal, or, with tax and service and discounts, to the total. A misread digit almost never cancels out. So:
- Lines that cannot be food go first. A "price" with a time, a date or a word in it is not an amount; a "price" bigger than the whole bill is a phone number or a zip code.
- When the lines add up, they are very probably all right, and the app says so.
- When exactly one change makes them add up, the app makes it and tells you: leaving out one line (the wine bar's address), or multiplying out a quantity it read as a price for one. Only when the arithmetic leaves exactly one option, and never down to a single line (that one is usually the subtotal itself).
- When nothing makes them add up, it says so ("the lines come to 21.50, the total says 24.30"), and if one amount is missing, offers it as a line with one tap.
The second model, and why the check makes two models better than one
Donut alone on American receipts got every line right on 3% of them as read, and 21% after the check. Most still did not add up. So I added a second, very different reader: Florence-2 (Microsoft, MIT), whose OCR returns every piece of text and where it is. About a hundred lines of JavaScript turn those pieces into rows (a name on the left, an amount on the right) and read the labelled rows at the bottom: subtotal, tax, total, and card or cash lines to ignore.
The two fail differently. Donut skips lines and invents dishes out of addresses; the OCR reads every line but gets a digit wrong now and then ("8.30" for "8.50"). Neither knows when it is wrong. The receipt does. So when Donut's read does not add up, Florence-2 reads the photo too, and the app keeps whichever read adds up. No confidence scores, no learned ensemble: arithmetic.
Measured
I wrote every rule while looking at 150 of WildReceipt's training receipts, then froze them and ran the test: WildReceipt's 472 test receipts, and the 100 test receipts of CORD, the kind Donut was trained on. A receipt's lines count as right when its set of prices matches the annotators' exactly.
WildReceipt test (436 receipts with every price annotated):
| Lines shown that are right | Real lines found | Receipts with every line right | Green "adds up" | …and every line right | |
|---|---|---|---|---|---|
| Donut, as read | 51% | 51% | 3% | ||
| Donut + the check | 77% | 50% | 21% | 83 receipts | 93% |
| Donut + Florence-2 + the check | 75% | 65% | 33% | 131 receipts | 93% |
CORD test (99 receipts): Donut is excellent at home. 83 receipts get the green tick, and all 83 are right line for line; 88% of receipts come out completely right. The second look costs a little here: when neither read adds up, the app keeps the read with more lines, and on Indonesian receipts that is sometimes Florence-2's weaker one (94% of shown lines right instead of 99%).
What I take from it:
- The green tick is the product. On receipts Donut was never trained on and the rules were never tuned on, when the app says it adds up, every line is right 93% of the time.
- The second model is what makes it lazy. It did not make the tick more reliable; it made it appear on 131 receipts instead of 83.
- The warning is honest too. When the app says the lines do not add up, only 8% of those receipts are actually completely right. Check the photo.
- A third of American receipts are still hard. Faded thermal paper, photos at an angle, 350-pixel images: neither model can read what is not legible. The app's job there is to say so, show how far off it is, and put the photo next to the lines.
The browser is not Node
Every number above comes from Node, where the test re-runs in seconds from a cache. The page runs the same JavaScript and the same model files, but the browser decodes the photo itself and runs the models on ONNX Runtime's WebAssembly build, and Donut's word-by-word decoding amplifies tiny differences. So I ran the page's own steps in Chrome on the first 50 test receipts. Not one read was word for word the same as Node's. Every line was right on 23% of them (Node, on the same receipts: 36%), and the green tick was right 10 times out of 12 (Node: 15 of 16). A small sample, but a real gap, and the harness to measure it is in the repo (node scripts/browser.mjs).
Why Does Open Innovation Matter?
A receipt is personal data. It has the last digits of a card, where you were, when, and often who served you. With the models running in the tab, "the photo never leaves your phone" is not a policy, it is where the code runs. Open the network tab, read a receipt, and watch nothing go out.
It works where receipts happen. Restaurants are basements, festivals and foreign countries without a data plan. Once the models are cached, Lazy Susan works in airplane mode.
Two open models are better than one, and I could only find that out because they are open. The trick that makes this work, letting the arithmetic choose between two readers, needs both readers on the same device, at no cost per read, with their outputs in my hands. With two hosted vision APIs, every receipt that does not add up would cost a second paid call and a second upload.
The 93% is a number about specific files. These weights, this quantisation, this decoder. A hosted model can change behind its endpoint on any day, and a number like that would quietly stop being true. These files will be the same next year.
Open data made it honest. WildReceipt (Apache-2.0) and CORD (CC BY 4.0) are hundreds of real receipts that people annotated line by line and gave away. Without them, "it works on my receipts" would be all I could tell you.
My Agent Session
I built this with Claude Code as a pair programmer, and did not save the session to DevRelay, so here is the next best thing: every number in this post can be re-run.
git clone https://github.com/VanshajPoonia/lazy-susan
cd lazy-susan && npm install && npm test
That downloads both datasets, reads the test receipts with Donut, and with Florence-2 where Donut's read does not add up (a couple of hours on an M4 the first time, cached after), and prints the tables above. SET=wild-tune npm test shows the 150 receipts the rules were written on.
The Lazy Payoff
A WildReceipt receipt has 5 lines on average. Typing just their prices into a bill-splitting app is 24 keystrokes, before a single dish name, a person or a tip. With Lazy Susan it is one photo and one tap per dish:
- on a third of American receipts it needs no typing at all, and on the green ones it is right 93% of the time;
- on Indonesian receipts like the ones Donut learned from, it is 88%;
- on the rest it tells you exactly what is off and by how much, next to the photo, so you fix a number or two instead of typing them all;
- and nobody does the maths: tax, service and tip are shared in proportion, and the shares add up to the total to the cent.
Prize Categories
None: Lazy Susan uses Donut, Florence-2 and transformers.js, and none of the partner technologies.
Top comments (0)