Only two lines on an Alberta power bill matter. A local model finds them.

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle.

What I Built

I run a small utility retailer in Alberta. Many new customer starts the same way: someone looks at their current bill, works out whether switching would save them money, and then types the site IDs, name and address into an enrolment form. A referral partner can bring in hundreds of homes at a time, and many of those homes have two sites, one for power and one for gas. That is a lot of squinting at photographs of bills.

bill2quote turns a photo or PDF of a utility bill into two things:

1. A one-page savings comparison the customer can read: what this bill cost on their current plan, what the same usage would cost on our offer, and the difference. If their current plan is cheaper, the page says that, plainly.
2. An enrolment-ready row with site ID(s), service address, name and current retailer, plus the partner's referral code.

A vision model running on my own machine reads the bill. Plain Python checks what it read, does every calculation, and writes the outputs. A person confirms the numbers on a review screen before any customer-facing quote exists.

The insight that makes this tractable: of the dozens of lines on an Alberta bill, only two belong to the retailer, the energy charge and the admin fee. Delivery, riders and the local access fee are set by regulators and municipalities and are the same whichever retailer you pick (Utilities Consumer Advocate, "How to read your bill"). So the comparison is two lines, plus GST on the difference. The arithmetic is easy. Reading the bill is the hard part.

Demo

The clip is the real flow on my Pixel 7, with the slow parts sped up. I photograph a printed bill, a local model reads it, and the review screen shows what doesn't add up. Two red checks block the quote until I fix the numbers, then the comparison appears, and "Edit numbers and re-run" fixes a last warning. The final screen is the enrolment list.

N.B.: Every bill in this post and in the repo is synthetic. The retailer, people, addresses, account numbers and site IDs are all invented.

Code

minchinweb / hacktoberfest-2026-2

Hacktoberfest 2026 Open-Source AI Challenge, Week 2: Side Hustle

bill2quote

Hacktoberfest 2026 Open-Source AI Challenge, Week 2: Side Hustle.

A photo or PDF of an Alberta utility bill in; an honest one-page savings comparison and an enrolment-ready record out. A local open-weight vision model (via Ollama) reads the bill Plain Python checks the arithmetic, does the money math and writes the outputs. A person confirms the numbers on a review screen before any customer-facing quote exists.

Only two lines on the bill belong to the retailer: the energy charge and the admin fee. Delivery riders and the local access fee are the same whichever retailer the customer picks, so savings come only from those two lines (plus GST on the difference). If the customer's current plan is cheaper, the page says so.

Data note

Every bill in this repo is synthetic. The retailer, people, addresses, account numbers and site IDs (0010099… electricity, 0000099… gas) are fictional, and the…

View on GitHub

Python 3.11+, FastAPI, Jinja2, pypdfium2, Pillow, Ollama. No build step, no database, no hosted API. bill2quote batch DIR for the office, bill2quote serve for the phone.

How I Built It

The model reads. The code checks.

The rule I wrote at the top of the spec and kept to: the model transcribes what is printed and never calculates, converts or corrects anything. A suspect number is flagged and shown to a person, never quietly fixed.

The model gets one request per bill with every page attached, a JSON schema, temperature 0 and thinking off (to encourage deterministic processing). Every field in the schema is required but nullable, so the output is either complete or rejected, and "I couldn't read it" is an answer the model is allowed to give. That is a lesson from Week 1, where every field was required too, and where the most useful failure was a garbled odometer reading that the pipeline happily booked as a plausible 642 km trip.

Then 20 deterministic checks run, each returning a flag with a severity. A few of them:

• C01: does the energy charge equal usage × rate? A small gap is a warning, since a tiered rate can explain it. More than three times apart blocks the quote, since nothing explains that.
• C04: do the meter readings add up to the usage?
• C06: is the rate plausible for its unit? (This one catches ¢ vs $ mix-ups and blocks the quote.)
• C07 / C17: is the site ID 13 digits starting with 00 (electricity, never 000) or 000 (gas), and is it not the account number? Some account numbers are also 13 digits, so a model can easily put the wrong one in the wrong box.
• C12: is anything quote-critical missing?
• C20: do the lines add up? Energy charge + admin fee + delivery should equal the printed subtotal. A value read from the wrong row breaks that straight away.
• C19: does the comparison itself make sense? If the two retailer lines on the offer come out three times higher or lower than on the current plan, a number was misread, and the quote is not shown.

Blocking flags stop a customer quote. Warnings must be acknowledged on the review screen with a tick box. Every edit a reviewer makes is logged with the model's value, the new value and the time.

How it did

I generated eight synthetic bills across three layouts (a one-page table, a summary plus detail page, and a two-column dual-fuel statement) with varied wording, plus a Pillow "phone photo" of one of them: rotated, in perspective, shadowed, JPEG at quality 60. Ground truth comes from the same data that rendered each bill. Two models, same prompt, same bills:

Model	Bills with every quote-critical field right	Valid output	Processing time per bill
Gemma 4 e4b	5 of 8	8 of 8	about 9s
Qwen3-VL 8b	7 of 8	7 of 8	about 73s

"Quote-critical" means usage, days, energy charge or rate, and admin fee. Field-level accuracy was 95% for Gemma and 97% for Qwen, which sounds great and is exactly the wrong number to look at: one wrong digit in the one field that matters is a wrong quote.

One failure, told straight. On an earlier version of my synthetic gas bill, Gemma put the usage (37.49 GJ) into the energy charge field. (That version had a bug of mine: the generator printed "Electricity Supply" on the energy line of a gas bill. With the label fixed, Gemma reads that PDF correctly, so a wrong label alone was enough to derail it. The misread still shows up on the photographed print, which predates the fix.) The correct charge was $183.33. A quote built from that would have told a customer they'd lose $117.70 by switching, when they'd have saved money. Nothing about it looks wrong at a glance. What caught it was check C01: charge 37.49 ≠ usage 37.49 × rate 4.89 = 183.33. (At that point C01 was only a warning; it now blocks the quote, see below.) Two more, from the current run: on the phone photo it read usage as 30.937 kWh, which looks like the previous meter reading (30,937) with the comma taken for a decimal point, and C04 flagged it; on another bill it labelled a 12.05 ¢/kWh rate as dollars, and C06 blocked the quote. Another bill came back with both site IDs one digit short: Gemma read 0010099000016 as 001009900016, and the gas ID on the same bill lost a zero the same way, apparently dropping one from a run of zeros. C07 blocked enrolment on both, with site ID is not 13 digits starting with 00. A truncated ID would be quietly accepted by any tool that only checks the field is filled in. Three of eight bills needed a human fix. That is why the review screen is the product, not an afterthought.

Real phone photos

I then printed the bills and photographed them with a Pixel 7 on a table: some tilted, one with my finger over part of the page, bill 03 as two photos. Seven bills, scored separately from the table above, which still includes the simulated photo. Same prompt, same models:

Model	Bills with every quote-critical field right	Valid output	Processing time per bill
Gemma 4 e4b	3 of 7	7 of 7	about 10
Qwen3-VL 8b	5 of 7	5 of 7	about 72

This is the hardest test so far, and Gemma lost ground: 3 of 7, against 5 of 8 on the clean PDFs. On the bills with a different label wording it often put values against the wrong row. My guess is that the tilt of the page shifts the right-hand values against the left-hand labels, but I haven't tested that. Its four failures were bills 02, 05, 06 and 08. On a second read of those four, a check fired on every one: C06 blocked the gas bill, where a usage of 37.49 had landed in the rate field, C12 blocked the one where usage came back empty, and the others raised C01, C03 or C04 warnings (for example meter reads give 0.700 kWh but usage says 700.0). Qwen got both of its misses by returning nothing usable: on bills 05 and 08 the JSON was cut off.

A warning was not enough. While recording the demo on bill 02, I fixed the fields the checks had flagged and left two that weren't, usage (the m³ figure had landed in the GJ field) and the energy charge, then ticked the box and confirmed. The page cheerfully said my current plan was cheaper by $4,072.21, or $47,947 over a year. C01 had only been a warning, and I had acknowledged it. So I changed two things. A C01 gap of more than 3× now blocks the quote, and a new check (C19) refuses any comparison whose two sides are more than 3× apart. The same data now gets a red field on the review screen and no quote. And the quote page has an Edit numbers and re-run link that reopens the form with the saved values and the offer choices, so fixing a number and re-running the comparison takes seconds. The new blocks can be overridden, since a customer on a very low promotional rate really could be more than 3× away, but only with a written reason that is saved with the bill, and the override stops applying the moment a number or the offer changes.

Qwen is ahead on both sets, but it always thinks (the think: false flag has no effect on it), it is about eight times slower, and one of its eight PDF outputs failed to validate (two did in the previous run, on different bills). I suspect the thinking ate the whole 8,192-token budget, but I haven't proved it. I kept Gemma as the default, but less comfortably than before: Qwen is two bills ahead on both sets (7 of 8 against 5 of 8 on the PDFs and the simulated photo, 5 of 7 against 3 of 7 on the real photos), though that is still a small sample and it is 8 times slower. Gemma is fast enough to use at a kitchen table, and when it is wrong the checks have caught it every time I looked. That is the trade I'm making, and more bills could change it. The reader is one config value (models.extract) and bill2quote eval --model X scores any Ollama vision model on the same bills, so swapping in a better model when one is released, or a locally fine-tuned one, means changing a setting and re-running the eval, not touching the code.

Honest limits

• Every bill here is synthetic, printed and photographed. None is a real customer's bill. Real retailer layouts are the next test -- all different layouts, not nearly as clear (colour and contrast), extra text (up ot 7 pages each) -- and I expect them to be harder. Seven photos is a small sample. The photos are in samples/photos/, and the results are in docs/eval/photos/.
• The printed bills I photographed were made before I fixed the generator's "Electricity Supply" label on gas bills, so photos 02 and 08 carry the old label. The PDF eval uses the fixed bills.
• A frontier hosted model would probably read a blurry phone photo better. That is a real cost of staying local.
• The 12-month estimate for gas is a rough days-scaling, labelled as such on the page, until I add heating-degree-day data. Per-bill numbers get top billing for that reason.
• The offers in the demo are made-up round numbers. This is not a statement about any real price.

Why Does Open Innovation Matter?

• The data is sensitive in a specific way. A bill carries a name, a service address, an account number, and a site ID. A site ID is the "address", the identity in the market and most of what's needed to enrol a site. Sending a stack of those to a hosted API is the wrong default for a business that handles thousands. Here the bills never leave the machine.
• No meter running. Reading a thousand multi-page bills costs nothing per page, and re-reading them after a prompt change is free. The eval above would have been a line item on an API invoice.
• The model is a config value, and I can measure it. Swapping Gemma for Qwen was one flag, and the table above is the result. The choice of model rests on numbers, not on a vendor's say-so.
• Room to specialise. Alberta bills are a narrow layout family. That is the kind of place a small model could later be fine-tuned. Not this week, but it's possible, which is the point.

My Agent Session

I built this with Claude Code from a written spec, in two sessions. The first took the spec to a working pipeline: scaffold, checks, ingest, extraction, quote math, the web review flow and tests.

bill2quote, part 1: from a written spec to a working pipeline in one session

You

Build the @spec.md

Agent

I'll start with M0 (scaffold), then M1, following the kickoff prompt. First I'll check the repo and toolchain.

▸ PowerShell ls -Force; python --version; pip list

Input

ls -Force; python --version; pip list

Output

Python 3.15.0; repo has .git, README.md, SPEC.md

Writing the first-commit files: .gitignore (per rule 3), licence, and pyproject.toml.

▸ Bash mkdir -p src/bill2quote/web/templates prompts samples tests docs; cat > .giti...

Input

$ mkdir -p src/bill2quote/web/templates prompts samples tests docs; cat > .gitignore

Output

(no output)

Dependencies installed. Now the core: schema and config.

▸ Bash cat > src/bill2quote/__init__.py, schema.py, config.py

Input

$ cat > src/bill2quote/__init__.py, schema.py, config.py

Output

(no output)

Now the deterministic checks (§9).

▸ Bash cat > checks.py <<'EOF'

Input

$ cat > checks.py <<'EOF'

Output

Exit code 2: unexpected EOF while looking for matching quote

The shell choked on that heredoc, so I'll use the Write tool for the files instead.

▸ Write src/bill2quote/checks.py

Input

src/bill2quote/checks.py

Output

File created successfully

Now tests for the checks (pass, fail, and missing-value cases).

▸ Write tests/test_checks.py

Input

tests/test_checks.py

Output

File created successfully

▸ Bash pytest -q

Input

$ pytest -q

Output

33 passed in 0.29s

33 checks tests pass. Now ingest, extract, the prompt, and the CLI with doctor.

▸ Write ingest.py, prompts/extract_v1.md, extract.py, doctor.py, cli.py

Input

ingest.py, prompts/extract_v1.md, extract.py, doctor.py, cli.py

Output

Files created

Now the synthetic sample generator: 8 bills across 3 layouts with varied wording, truth files built from the same data, plus a Pillow "phone photo" for #4.

▸ Write samples/make_samples.py

Input

samples/make_samples.py

Output

File created successfully

▸ Bash python samples/make_samples.py

Input

$ python samples/make_samples.py

Output

wrote 01_elec_clean.pdf, 02_gas_clean.pdf, 03_dual …

The generator works (it needs numpy, so I'll add that to the dev deps). Next I'll check that every truth file passes the checks as expected, then look at the output.

▸ Bash pytest -q

Input

$ pytest -q

Output

42 passed

All eight truths behave as designed. I'll look at what the bills actually look like.

▸ Read samples/04_phone_photo_dual_fuel.jpg

Input

samples/04_phone_photo_dual_fuel.jpg

Output

[image omitted]

The page content is legible, but the shadow is wrong: a hard diagonal band from the rotated gradient, not a soft shadow. I'll replace it with a smooth numpy gradient.

▸ Edit samples/make_samples.py

Input

samples/make_samples.py

Output

updated

The photo now reads like a phone shot. Next I'll clean up the lint, run the tests, and try doctor against Ollama.

▸ Bash ruff check --fix; pytest -q

Input

$ ruff check --fix; pytest -q

Output

Found 1 error (1 fixed)

Ollama isn't running here, so I can't test model reading yet. I'll commit what's done.

▸ Bash git add -A && git commit -m "M0 scaffold + M1 (partial)"

Input

$ git add -A && git commit -m "M0 scaffold + M1 (partial)"

Output

committed 2da9267

I've built M0 and most of M1, and committed it as 2da9267, but I haven't tested reading a bill with a model yet. Ollama isn't running on this machine, and bill2quote doctor fails with connection refused.

What works
- Scaffold: .gitignore, MIT licence, pyproject.toml, and a .venv. The CLI has init, doctor, read and check.
- Reading code: the schema, config loader, image/PDF ingest, and the Ollama call (schema-constrained, temperature 0, thinking off). Written but unrun against a model.
- Checks: C01–C18, each with tests for the pass, fail and missing-value cases.
- Samples: 8 bills with truth files, in three layouts with varied wording. Bill #4 is a Pillow-made phone photo of #3. Every truth file raises exactly the flags it should.

Show more

You

Ollama should now be up. Carry on

Agent

▸ Bash bill2quote doctor; ollama list

Input

$ bill2quote doctor; ollama list

Output

Ollama reachable; only gemma4:e4b is pulled

qwen3-vl:8b isn't pulled, although the spec says it is. Eval needs a second model, so I'll pull it in the background and start reading bills with Gemma meanwhile.

▸ Bash ollama pull qwen3-vl:8b (background); bill2quote read samples/01_elec_clean.pdf

Input

$ ollama pull qwen3-vl:8b (background); bill2quote read samples/01_elec_clean.pdf

Output

real 0m8.396s

Gemma read bill #1 in 8 seconds. Now eval, which scores each model field by field against the truth files, and also gives the M1 check for all 8 bills.

▸ Write src/bill2quote/evaluate.py

Input

src/bill2quote/evaluate.py

Output

File created

All 8 bills come back as valid BillFacts, which meets the M1 gate. Quote-critical accuracy is 5/8, so I'll look at what went wrong on the three misses.

Qwen failed all 8, with 0 valid reads. I'll read the actual error before treating that as a result.

▸ Bash print the error for each Qwen result

Input

$ print the error for each Qwen result

Output

model output did not match the schema: 1 validation error

Qwen returns empty content. I'll look at the raw response to see where its output is going.

▸ Bash raw request to Ollama /api/chat with Qwen

Input

$ raw request to Ollama /api/chat with Qwen

Output

content: '' | thinking: 1815 chars | done_reason: length at 637 tokens

Qwen always emits thinking (the think:false flag has no effect on it) and is cut off with done_reason: length at about 637 tokens. I'll try a bigger context and output budget.

▸ Bash same request with num_ctx and num_predict raised

Input

$ same request with num_ctx and num_predict raised

Output

51 s, done_reason: stop, content: valid JSON

That works with the bigger budget, at 51 s per bill because of the thinking. I'll make num_ctx and num_predict config values, applied to every model.

▸ Write annualize.py, quote.py, tests/test_quote.py

Input

annualize.py, quote.py, tests/test_quote.py

Output

Files created

▸ Bash pytest -q

Input

$ pytest -q

Output

49 passed

The 49 tests pass, and the hand calculation (79.32 − 64.0188 = 15.30) matches. Bill #8 is negative with the demo offers. Now the quote page renderer, enrolment CSV and batch.

▸ Write quote.html, render.py, enrol.py, store.py

Input

quote.html, render.py, enrol.py, store.py

Output

Files created

The background eval is still running. In the meantime I'll write the web review flow (M3): upload, polling, review with flags, edit log, quote and enrol.

▸ Write src/bill2quote/web/app.py

Input

src/bill2quote/web/app.py

Output

File created

flags_for is more convoluted than it needs to be. I'll make duplicate detection simply exclude the bill's own folder.

Now the templates: base styles, upload, polling wait page, review, quote bar, enrolments and login.

The shell choked on a heredoc again, so I'll write these with the Write tool. Writing each template directly.

▸ Write base.html, login.html, index.html, wait.html, review.html, quote_bar.html, en...

Input

base.html, login.html, index.html, wait.html, review.html, quote_bar.html, enrolments.html

Output

Files created

Next, a test for the whole web flow without a model: seed a bill from truth #7 (the planted error), then go through review, the edit log, quote and enrol.

▸ Write tests/test_web.py

Input

tests/test_web.py

Output

6 passed

All six pass, but two of my assertions are loose (they have or clauses that can't fail). I'll tighten them so they test what they claim.

Eval finished. Gemma is 5/8 quote-critical in 6.7 s per bill. Qwen is 6/8 but takes 78 s per bill, and 2 of its 8 outputs weren't valid. I'll see why those two failed before drawing conclusions.

The checks caught each real misread: a meter reading taken as usage, a ¢/$ unit mix-up, and a missing usage value. I'll commit, then smoke-test the live server with a real upload.

The review page renders, but the page image returns 404. I'll check on disk.

That's a real bug. This bill had already been read by batch, so upload skipped the background job, and page images are only written by that job. I'll make upload always ensure the pages exist.

I'll add a regression test for that, then restart the server and run the whole flow (review → quote → enrol) through real HTTP.

▸ Bash pytest -q; run the flow against the live server

Input

$ pytest -q; run the flow against the live server

Output

56 passed; review 200, quote 200, enrolment row written

M0 to M3 are built and committed (latest 76f2c31). All 56 tests pass. I also ran the whole flow against the real server and Gemma. The phone run on your Pixel 7 is still to do, so M3 isn't fully done.

What now works
- read, check, batch, eval and serve, along with doctor.
- Quote math: checked by hand. For bill #1 the retailer lines come to $79.32 now against $64.0188 on the offer, a difference of $15.30, which is $16.07 with GST.
- Customer view: masks site IDs and hides flags.
- Negative savings: bill #8 shows "your current plan is cheaper" with the demo offers.
- Web flow over real HTTP: upload, review, confirm, the customer quote, and an enrolment row with no account number.
- Bug found and fixed: a bill that had already been read skipped creating its page images. There's a regression test.

Model eval (8 synthetic bills)
- Gemma 4 e4b: 5 of 8 quote-critical correct, all 8 valid, about 7 s a bill.
- Qwen3-VL 8b: 6 of 8, 2 outputs invalid, about 78 s a bill.

Show more

You

Re "Batch duplicate flag: batch doesn't run the duplicate-bill check (C16). The web app does." Fix this. Also draft the Dev post

Agent

Fixing C16 in batch first. I'll also show flag IDs rather than counts in the flags column, so the duplicate flag is visible.

▸ Bash edit cli.py; pytest -q

Input

$ edit cli.py; pytest -q

Output

58 passed

▸ Write POST.md

Input

POST.md

Output

File created

▸ Bash git commit -m "POST: correct claims about the phone photo and hardware"

Input

$ git commit -m "POST: correct claims about the phone photo and hardware"

Output

committed 1ccf427

batch now runs the duplicate check, and the post is drafted in POST.md. 58 tests pass and both changes are committed.

C16 in batch
- It flags a bill whose site and period match a different bill already processed.
- On the samples it now flags #3 and #4, which are the same bill as a PDF and as a phone photo.
- A bill never flags against its own folder.
- The flags column now lists flag IDs (like C01 C16) instead of counts. There are two new tests, with no model needed.

The post
- It follows the DEV template headings in order, with the intro line and the four tags.
- The hook, the "two lines" insight, how the checks work, the eval table, one failure story told straight, limits, why open, and the Author's Note are all written from what's in the repo.
- The failure story is the Gemma gas bill. It put the usage into the charge field, which would have told the customer they'd lose $117.70 when they'd actually have saved money. Check C01 caught it.
- Marked [TODO] in the post: the demo, the first-user timings, and the real phone photos.

Show more

6 of 6 messages

The second is the last two days: real phone photos, the plausibility checks, the override, and the demo.

bill2quote, part 2: real phone photos, plausibility checks and the demo

You

Fix the site ID coding to match the SPEC. An electricity Site ID will not start 000. Regenerate the synthetic bills and related testing.

Agent

Electricity site IDs must now be 13 digits starting 00 (never 000), gas IDs 13 digits starting 000. I changed the regexes in config.py and config.example.toml, made the C07 message name the right prefix per commodity, regenerated the synthetic bills, and added a test for both prefixes. 60 tests pass.

▸ Edit/Bash config.py, checks.py, make_samples.py, tests

Input

config.py, checks.py, make_samples.py, tests

Output

60 passed

You

Re-run the eval on the regenerated bills and re-read the worked examples in the post.

Agent

Eval on the regenerated PDFs: Gemma 4 e4b 5 of 8 quote-critical, 8 of 8 valid, about 9 s a bill. Qwen3-VL 8b ahead, about 73 s a bill. I re-read all three worked examples against fresh runs and they still reproduce. Added a C07 example (Gemma dropped a digit from the site IDs and C07 blocked enrolment).

You

I printed the bills and photographed them with my phone. Please remove the location from EXIF if we commit them. Crop the slippers out of two of them; the hand is fine.

Agent

All eight photos had GPS coordinates. I re-saved each from pixels only, so no EXIF or GPS survives, cropped the slippers out of photos 02 and 08, and extended the eval to accept multi-photo bills (bill 03 is two photos). My first patch to evaluate.py applied only halfway because the file uses Windows line endings, so the first eval run crashed; I fixed the loop and re-ran.

Result on 7 real photos: Gemma 3 of 7 (all valid), Qwen 5 of 7 (only 5 valid). A check fired on every Gemma miss.

You

Stick with Gemma for now, but note the project is designed so models can be swapped as better ones are released or fine tuned locally.

Agent

Added a sentence to the post: the reader is one config value (models.extract), and bill2quote eval --model X scores any Ollama vision model on the same bills.

You

Next is the GIF demo. Review my screen recording, trim it, add subtitles, make an 8 FPS GIF.

Agent

I dumped frames at a constant rate (the recording is variable frame rate, which made seeking unreliable), cut segments with speed-ups for the slow waits, covered the browser address bar, burned in captions, dropped partial and dimmed frames, and encoded at 8 fps. A first pass leaked the LAN address in a dimmed file-chooser frame and an automated scan caught it, so those frames are skipped.

You

In the recording I confirmed bill 02 and it said my current plan was cheaper by $4,072. Fix that, and allow returning to the field entry form to fix numbers and re-run the comparison.

Agent

Root cause: I had fixed the flagged fields but left a wrong usage (the m3 figure in the GJ field) and a wrong energy charge, and C01 was only a warning I had acknowledged. Changes: a C01 gap over 3x now blocks the quote; a new check C19 refuses any comparison whose two retailer-line sides are more than 3x apart; the quote page gets an 'Edit numbers and re-run' link that reopens the form with the saved values and offer choices. On the real session data the $4,072 quote is now refused and the energy charge shows a red C01.

You

Allow the review override option. Add the check on addition.

Agent

Added assess.py, shared by the web app and the CLI. Only C01-block and C19 can be overridden, a written reason is required and logged, and the override stops applying as soon as a number or the offer changes. Added C20: energy charge + admin fee + delivery must equal the printed subtotal. A stray newline from my patch script broke a string literal in app.py and I fixed it directly. 72 tests pass.

You

On Bill 2 the line reads 'Electricity Supply' but this is a gas bill.

Agent

That was a bug in my sample generator: the alternate wording reused the electricity label for the gas energy line. I gave gas its own label ('Natural Gas Supply'), regenerated, and re-ran the eval. The post now says the printed photos predate the fix. Then: fresh-clone test (install, 72 tests, doctor, serve), and found git's autocrlf had altered the PDFs, so I added .gitattributes marking them binary and verified a clean GitHub clone byte-exact.

You

Use the video times for the tool time. My manual time was 17:44 for 7 bills. I caught a $25 promotional discount and confirmed no bill carried a balance forward. Value to me is as part of a bigger CRM package.

Agent

Wrote up the first-user session in docs/first-user.md and the post: manual about 2:32 a bill; tool about 1:02 on a clean read and 3:09 on a bad one, measured to the first correct quote so enrolment is excluded on both sides. The post says plainly that the tool never reads the previous balance, and that the CRM link is not built.

You

Add a Markdown link to our Week 1 post, add the link to the GIF demo, add the agent sessions, remove the top comment.

Agent

Done in POST.md. I also corrected one sentence to match what the Week 1 post actually says (every field required; the most useful failure was a garbled odometer booked as a plausible 642 km trip).

20 of 20 messages

Two moments worth pointing at. In the first session, Qwen came back with empty output; reading the raw response showed its thinking was eating the whole token budget, so the fix was a bigger budget, not a prompt change. In the second, I confirmed a bill that still had a wrong usage and energy charge and got a quote saying my current plan was cheaper by $4,072. That is where C19 and the override came from.

My First User

My first user is me, as the retailer: internal use by the buyer, which is a weaker test than an outside customer, but the retailer is the one who pays. The full notes are in docs/first-user.md.

I reviewed the seven printed bills by hand and quoted the comparison, then ran the tool on two of them. Enrolment is extra time in both cases and isn't counted in either case, since I have no way to automate it yet. All seven bills are in one basic layout with no extra text, so reading them by eye is easy and fast. That flatters the manual side.

• Manual: 17 minutes 44 seconds for seven bills, about 2 min 32 s each.
• With bill2quote: about 1 min 02 s for a bill that read cleanly, and about 3 min 09 s for one the model misread badly and I had to correct field by field. The mean of those two is about 2 min 06 s. Those were demo recordings, not a stopwatch session, so I'm not making a speed claim beyond this: a little faster on average, much faster when the read is clean, slower when it isn't.
• What I caught that the tool didn't: a $25 promotional discount on one bill, which I need to discuss with the client, and that none of the bills carried a balance forward. A balance carried forward is a credit-risk signal, and we might not want that customer. The tool reads credits and lists them as an informational note, but it never reads the previous balance at all.
• What the tool flags that I missed: I did not see any of the issues it would have flagged. On these bills that is the planted $20 error in the energy charge on bill 07 (C01), the estimated meter read on bill 05 and the credit on bill 06. I did not run bill 07 through the tool live, so that catch comes from the tests and the eval, not from the session.
• Verdict as the buyer: as a stand-alone tool its value is hard to pin down. For me it would be part of a bigger package: launching a CRM. A CRM that does this wins over one that doesn't, and no CRM does it out of the box, so the value is as a differentiator inside a package deal, not a price per bill. What I expect from it: it can deal with different bill formats once properly configured, whereas this test was one format I could read by eye; a properly formatted quote for the client instead of my scribbles on their bill; and a record of each enrolment (partner code, bill, offer) that can go into a CRM for follow-up and to confirm the enrolment went through. The quote and the record exist today. The CRM connection and the success tracking do not, and neither does reading the previous balance or treating a promotional credit as a conversation with the client.

Prize Categories

Best Use of Gemma. Gemma 4 e4b is the default reader. It reads every bill in the eval to a valid result and takes about nine seconds a bill on my Windows laptop. I report where it fails and how the checks catch it, and I report that Qwen3-VL scored two bills higher on both the PDFs and the real photos. I'm not claiming any other category.

---

Author's Notes

I drafted this post with AI help, as I did the earlier entries, and edited it myself. The software was built with Claude Code from a spec I wrote with it. All numbers above come from runs saved in the repo (docs/eval/).