This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle.

What I Built

I run a small utility retailer in Alberta. Many new customer starts the same way: someone looks at their current bill, works out whether switching would save them money, and then types the site IDs, name and address into an enrolment form. A referral partner can bring in hundreds of homes at a time, and many of those homes have two sites, one for power and one for gas. That is a lot of squinting at photographs of bills.

bill2quote turns a photo or PDF of a utility bill into two things:

  1. A one-page savings comparison the customer can read: what this bill cost on their current plan, what the same usage would cost on our offer, and the difference. If their current plan is cheaper, the page says that, plainly.
  2. An enrolment-ready row with site ID(s), service address, name and current retailer, plus the partner's referral code.

A vision model running on my own machine reads the bill. Plain Python checks what it read, does every calculation, and writes the outputs. A person confirms the numbers on a review screen before any customer-facing quote exists.

The insight that makes this tractable: of the dozens of lines on an Alberta bill, only two belong to the retailer, the energy charge and the admin fee. Delivery, riders and the local access fee are set by regulators and municipalities and are the same whichever retailer you pick (Utilities Consumer Advocate, "How to read your bill"). So the comparison is two lines, plus GST on the difference. The arithmetic is easy. Reading the bill is the hard part.

Demo

bill2quote on a phone: photo of a bill in, red checks, fix the numbers, quote out

The clip is the real flow on my Pixel 7, with the slow parts sped up. I photograph a printed bill, a local model reads it, and the review screen shows what doesn't add up. Two red checks block the quote until I fix the numbers, then the comparison appears, and "Edit numbers and re-run" fixes a last warning. The final screen is the enrolment list.

N.B.: Every bill in this post and in the repo is synthetic. The retailer, people, addresses, account numbers and site IDs are all invented.

Code

GitHub logo minchinweb / hacktoberfest-2026-2

Hacktoberfest 2026 Open-Source AI Challenge, Week 2: Side Hustle

bill2quote

Hacktoberfest 2026 Open-Source AI Challenge, Week 2: Side Hustle.

A photo or PDF of an Alberta utility bill in; an honest one-page savings comparison and an enrolment-ready record out. A local open-weight vision model (via Ollama) reads the bill Plain Python checks the arithmetic, does the money math and writes the outputs. A person confirms the numbers on a review screen before any customer-facing quote exists.

Only two lines on the bill belong to the retailer: the energy charge and the admin fee. Delivery riders and the local access fee are the same whichever retailer the customer picks, so savings come only from those two lines (plus GST on the difference). If the customer's current plan is cheaper, the page says so.

Data note

Every bill in this repo is synthetic. The retailer, people, addresses, account numbers and site IDs (0010099… electricity, 0000099… gas) are fictional, and the…

Python 3.11+, FastAPI, Jinja2, pypdfium2, Pillow, Ollama. No build step, no database, no hosted API. bill2quote batch DIR for the office, bill2quote serve for the phone.

How I Built It

The model reads. The code checks.

The rule I wrote at the top of the spec and kept to: the model transcribes what is printed and never calculates, converts or corrects anything. A suspect number is flagged and shown to a person, never quietly fixed.

The model gets one request per bill with every page attached, a JSON schema, temperature 0 and thinking off (to encourage deterministic processing). Every field in the schema is required but nullable, so the output is either complete or rejected, and "I couldn't read it" is an answer the model is allowed to give. That is a lesson from Week 1, where every field was required too, and where the most useful failure was a garbled odometer reading that the pipeline happily booked as a plausible 642 km trip.

Then 20 deterministic checks run, each returning a flag with a severity. A few of them:

  • C01: does the energy charge equal usage × rate? A small gap is a warning, since a tiered rate can explain it. More than three times apart blocks the quote, since nothing explains that.
  • C04: do the meter readings add up to the usage?
  • C06: is the rate plausible for its unit? (This one catches ¢ vs $ mix-ups and blocks the quote.)
  • C07 / C17: is the site ID 13 digits starting with 00 (electricity, never 000) or 000 (gas), and is it not the account number? Some account numbers are also 13 digits, so a model can easily put the wrong one in the wrong box.
  • C12: is anything quote-critical missing?
  • C20: do the lines add up? Energy charge + admin fee + delivery should equal the printed subtotal. A value read from the wrong row breaks that straight away.
  • C19: does the comparison itself make sense? If the two retailer lines on the offer come out three times higher or lower than on the current plan, a number was misread, and the quote is not shown.

Blocking flags stop a customer quote. Warnings must be acknowledged on the review screen with a tick box. Every edit a reviewer makes is logged with the model's value, the new value and the time.

How it did

I generated eight synthetic bills across three layouts (a one-page table, a summary plus detail page, and a two-column dual-fuel statement) with varied wording, plus a Pillow "phone photo" of one of them: rotated, in perspective, shadowed, JPEG at quality 60. Ground truth comes from the same data that rendered each bill. Two models, same prompt, same bills:

Model Bills with every quote-critical field right Valid output Processing time per bill
Gemma 4 e4b 5 of 8 8 of 8 about 9s
Qwen3-VL 8b 7 of 8 7 of 8 about 73s

"Quote-critical" means usage, days, energy charge or rate, and admin fee. Field-level accuracy was 95% for Gemma and 97% for Qwen, which sounds great and is exactly the wrong number to look at: one wrong digit in the one field that matters is a wrong quote.

One failure, told straight. On an earlier version of my synthetic gas bill, Gemma put the usage (37.49 GJ) into the energy charge field. (That version had a bug of mine: the generator printed "Electricity Supply" on the energy line of a gas bill. With the label fixed, Gemma reads that PDF correctly, so a wrong label alone was enough to derail it. The misread still shows up on the photographed print, which predates the fix.) The correct charge was $183.33. A quote built from that would have told a customer they'd lose $117.70 by switching, when they'd have saved money. Nothing about it looks wrong at a glance. What caught it was check C01: charge 37.49 ≠ usage 37.49 × rate 4.89 = 183.33. (At that point C01 was only a warning; it now blocks the quote, see below.) Two more, from the current run: on the phone photo it read usage as 30.937 kWh, which looks like the previous meter reading (30,937) with the comma taken for a decimal point, and C04 flagged it; on another bill it labelled a 12.05 ¢/kWh rate as dollars, and C06 blocked the quote. Another bill came back with both site IDs one digit short: Gemma read 0010099000016 as 001009900016, and the gas ID on the same bill lost a zero the same way, apparently dropping one from a run of zeros. C07 blocked enrolment on both, with site ID is not 13 digits starting with 00. A truncated ID would be quietly accepted by any tool that only checks the field is filled in. Three of eight bills needed a human fix. That is why the review screen is the product, not an afterthought.

Real phone photos

I then printed the bills and photographed them with a Pixel 7 on a table: some tilted, one with my finger over part of the page, bill 03 as two photos. Seven bills, scored separately from the table above, which still includes the simulated photo. Same prompt, same models:

Model Bills with every quote-critical field right Valid output Processing time per bill
Gemma 4 e4b 3 of 7 7 of 7 about 10
Qwen3-VL 8b 5 of 7 5 of 7 about 72

This is the hardest test so far, and Gemma lost ground: 3 of 7, against 5 of 8 on the clean PDFs. On the bills with a different label wording it often put values against the wrong row. My guess is that the tilt of the page shifts the right-hand values against the left-hand labels, but I haven't tested that. Its four failures were bills 02, 05, 06 and 08. On a second read of those four, a check fired on every one: C06 blocked the gas bill, where a usage of 37.49 had landed in the rate field, C12 blocked the one where usage came back empty, and the others raised C01, C03 or C04 warnings (for example meter reads give 0.700 kWh but usage says 700.0). Qwen got both of its misses by returning nothing usable: on bills 05 and 08 the JSON was cut off.

A warning was not enough. While recording the demo on bill 02, I fixed the fields the checks had flagged and left two that weren't, usage (the m³ figure had landed in the GJ field) and the energy charge, then ticked the box and confirmed. The page cheerfully said my current plan was cheaper by $4,072.21, or $47,947 over a year. C01 had only been a warning, and I had acknowledged it. So I changed two things. A C01 gap of more than 3× now blocks the quote, and a new check (C19) refuses any comparison whose two sides are more than 3× apart. The same data now gets a red field on the review screen and no quote. And the quote page has an Edit numbers and re-run link that reopens the form with the saved values and the offer choices, so fixing a number and re-running the comparison takes seconds. The new blocks can be overridden, since a customer on a very low promotional rate really could be more than 3× away, but only with a written reason that is saved with the bill, and the override stops applying the moment a number or the offer changes.

Qwen is ahead on both sets, but it always thinks (the think: false flag has no effect on it), it is about eight times slower, and one of its eight PDF outputs failed to validate (two did in the previous run, on different bills). I suspect the thinking ate the whole 8,192-token budget, but I haven't proved it. I kept Gemma as the default, but less comfortably than before: Qwen is two bills ahead on both sets (7 of 8 against 5 of 8 on the PDFs and the simulated photo, 5 of 7 against 3 of 7 on the real photos), though that is still a small sample and it is 8 times slower. Gemma is fast enough to use at a kitchen table, and when it is wrong the checks have caught it every time I looked. That is the trade I'm making, and more bills could change it. The reader is one config value (models.extract) and bill2quote eval --model X scores any Ollama vision model on the same bills, so swapping in a better model when one is released, or a locally fine-tuned one, means changing a setting and re-running the eval, not touching the code.

Honest limits

  • Every bill here is synthetic, printed and photographed. None is a real customer's bill. Real retailer layouts are the next test -- all different layouts, not nearly as clear (colour and contrast), extra text (up ot 7 pages each) -- and I expect them to be harder. Seven photos is a small sample. The photos are in samples/photos/, and the results are in docs/eval/photos/.
  • The printed bills I photographed were made before I fixed the generator's "Electricity Supply" label on gas bills, so photos 02 and 08 carry the old label. The PDF eval uses the fixed bills.
  • A frontier hosted model would probably read a blurry phone photo better. That is a real cost of staying local.
  • The 12-month estimate for gas is a rough days-scaling, labelled as such on the page, until I add heating-degree-day data. Per-bill numbers get top billing for that reason.
  • The offers in the demo are made-up round numbers. This is not a statement about any real price.

Why Does Open Innovation Matter?

  • The data is sensitive in a specific way. A bill carries a name, a service address, an account number, and a site ID. A site ID is the "address", the identity in the market and most of what's needed to enrol a site. Sending a stack of those to a hosted API is the wrong default for a business that handles thousands. Here the bills never leave the machine.
  • No meter running. Reading a thousand multi-page bills costs nothing per page, and re-reading them after a prompt change is free. The eval above would have been a line item on an API invoice.
  • The model is a config value, and I can measure it. Swapping Gemma for Qwen was one flag, and the table above is the result. The choice of model rests on numbers, not on a vendor's say-so.
  • Room to specialise. Alberta bills are a narrow layout family. That is the kind of place a small model could later be fine-tuned. Not this week, but it's possible, which is the point.

My Agent Session

I built this with Claude Code from a written spec, in two sessions. The first took the spec to a working pipeline: scaffold, checks, ingest, extraction, quote math, the web review flow and tests.

bill2quote, part 1: from a written spec to a working pipeline in one session
You

Build the @spec.md

Agent

I'll start with M0 (scaffold), then M1, following the kickoff prompt. First I'll check the repo and toolchain.

▸ PowerShell ls -Force; python --version; pip list
ls -Force; python --version; pip list
Python 3.15.0; repo has .git, README.md, SPEC.md

Writing the first-commit files: .gitignore (per rule 3), licence, and pyproject.toml.

▸ Bash mkdir -p src/bill2quote/web/templates prompts samples tests docs; cat > .giti...
$ mkdir -p src/bill2quote/web/templates prompts samples tests docs; cat > .gitignore
(no output)

Dependencies installed. Now the core: schema and config.

▸ Bash cat > src/bill2quote/__init__.py, schema.py, config.py
$ cat > src/bill2quote/__init__.py, schema.py, config.py
(no output)

Now the deterministic checks (§9).

▸ Bash cat > checks.py <<'EOF'
$ cat > checks.py <<'EOF'
Exit code 2: unexpected EOF while looking for matching quote

The shell choked on that heredoc, so I'll use the Write tool for the files instead.

▸ Write src/bill2quote/checks.py
src/bill2quote/checks.py
File created successfully

Now tests for the checks (pass, fail, and missing-value cases).

▸ Write tests/test_checks.py
tests/test_checks.py
File created successfully
▸ Bash pytest -q
$ pytest -q
33 passed in 0.29s

33 checks tests pass. Now ingest, extract, the prompt, and the CLI with doctor.

▸ Write ingest.py, prompts/extract_v1.md, extract.py, doctor.py, cli.py
ingest.py, prompts/extract_v1.md, extract.py, doctor.py, cli.py
Files created

Now the synthetic sample generator: 8 bills across 3 layouts with varied wording, truth files built from the same data, plus a Pillow "phone photo" for #4.

▸ Write samples/make_samples.py
samples/make_samples.py
File created successfully
▸ Bash python samples/make_samples.py
$ python samples/make_samples.py
wrote 01_elec_clean.pdf, 02_gas_clean.pdf, 03_dual …

The generator works (it needs numpy, so I'll add that to the dev deps). Next I'll check that every truth file passes the checks as expected, then look at the output.

▸ Bash pytest -q
$ pytest -q
42 passed

All eight truths behave as designed. I'll look at what the bills actually look like.

▸ Read samples/04_phone_photo_dual_fuel.jpg
samples/04_phone_photo_dual_fuel.jpg
[image omitted]

The page content is legible, but the shadow is wrong: a hard diagonal band from the rotated gradient, not a soft shadow. I'll replace it with a smooth numpy gradient.

▸ Edit samples/make_samples.py
samples/make_samples.py
updated

The photo now reads like a phone shot. Next I'll clean up the lint, run the tests, and try doctor against Ollama.

▸ Bash ruff check --fix; pytest -q
$ ruff check --fix; pytest -q
Found 1 error (1 fixed)

Ollama isn't running here, so I can't test model reading yet. I'll commit what's done.

▸ Bash git add -A && git commit -m "M0 scaffold + M1 (partial)"
$ git add -A && git commit -m "M0 scaffold + M1 (partial)"
committed 2da9267

I've built M0 and most of M1, and committed it as 2da9267, but I haven't tested reading a bill with a model yet. Ollama isn't running on this machine, and bill2quote doctor fails with connection refused.

What works
- Scaffold: .gitignore, MIT licence, pyproject.toml, and a .venv. The CLI has init, doctor, read and check.
- Reading code: the schema, config loader, image/PDF ingest, and the Ollama call (schema-constrained, temperature 0, thinking off). Written but unrun against a model.
- Checks: C01–C18, each with tests for the pass, fail and missing-value cases.
- Samples: 8 bills with truth files, in three layouts with varied wording. Bill #4 is a Pillow-made phone photo of #3. Every truth file raises exactly the flags it should.

Show more
You

Ollama should now be up. Carry on

Agent
▸ Bash bill2quote doctor; ollama list
$ bill2quote doctor; ollama list
Ollama reachable; only gemma4:e4b is pulled

qwen3-vl:8b isn't pulled, although the spec says it is. Eval needs a second model, so I'll pull it in the background and start reading bills with Gemma meanwhile.

▸ Bash ollama pull qwen3-vl:8b (background); bill2quote read samples/01_elec_clean.pdf
$ ollama pull qwen3-vl:8b (background); bill2quote read samples/01_elec_clean.pdf
real 0m8.396s

Gemma read bill #1 in 8 seconds. Now eval, which scores each model field by field against the truth files, and also gives the M1 check for all 8 bills.

▸ Write src/bill2quote/evaluate.py
src/bill2quote/evaluate.py
File created

All 8 bills come back as valid BillFacts, which meets the M1 gate. Quote-critical accuracy is 5/8, so I'll look at what went wrong on the three misses.

Qwen failed all 8, with 0 valid reads. I'll read the actual error before treating that as a result.

▸ Bash print the error for each Qwen result
$ print the error for each Qwen result
model output did not match the schema: 1 validation error

Qwen returns empty content. I'll look at the raw response to see where its output is going.

▸ Bash raw request to Ollama /api/chat with Qwen
$ raw request to Ollama /api/chat with Qwen
content: '' | thinking: 1815 chars | done_reason: length at 637 tokens

Qwen always emits thinking (the think:false flag has no effect on it) and is cut off with done_reason: length at about 637 tokens. I'll try a bigger context and output budget.

▸ Bash same request with num_ctx and num_predict raised
$ same request with num_ctx and num_predict raised
51 s, done_reason: stop, content: valid JSON

That works with the bigger budget, at 51 s per bill because of the thinking. I'll make num_ctx and num_predict config values, applied to every model.

▸ Write annualize.py, quote.py, tests/test_quote.py
annualize.py, quote.py, tests/test_quote.py
Files created
▸ Bash pytest -q
$ pytest -q
49 passed

The 49 tests pass, and the hand calculation (79.32 − 64.0188 = 15.30) matches. Bill #8 is negative with the demo offers. Now the quote page renderer, enrolment CSV and batch.

▸ Write quote.html, render.py, enrol.py, store.py
quote.html, render.py, enrol.py, store.py
Files created

The background eval is still running. In the meantime I'll write the web review flow (M3): upload, polling, review with flags, edit log, quote and enrol.

▸ Write src/bill2quote/web/app.py
src/bill2quote/web/app.py
File created

flags_for is more convoluted than it needs to be. I'll make duplicate detection simply exclude the bill's own folder.

Now the templates: base styles, upload, polling wait page, review, quote bar, enrolments and login.

The shell choked on a heredoc again, so I'll write these with the Write tool. Writing each template directly.

▸ Write base.html, login.html, index.html, wait.html, review.html, quote_bar.html, en...
base.html, login.html, index.html, wait.html, review.html, quote_bar.html, enrolments.html
Files created

Next, a test for the whole web flow without a model: seed a bill from truth #7 (the planted error), then go through review, the edit log, quote and enrol.

▸ Write tests/test_web.py
tests/test_web.py
6 passed

All six pass, but two of my assertions are loose (they have or clauses that can't fail). I'll tighten them so they test what they claim.

Eval finished. Gemma is 5/8 quote-critical in 6.7 s per bill. Qwen is 6/8 but takes 78 s per bill, and 2 of its 8 outputs weren't valid. I'll see why those two failed before drawing conclusions.

The checks caught each real misread: a meter reading taken as usage, a ¢/$ unit mix-up, and a missing usage value. I'll commit, then smoke-test the live server with a real upload.

The review page renders, but the page image returns 404. I'll check on disk.

That's a real bug. This bill had already been read by batch, so upload skipped the background job, and page images are only written by that job. I'll make upload always ensure the pages exist.

I'll add a regression test for that, then restart the server and run the whole flow (review → quote → enrol) through real HTTP.

▸ Bash pytest -q; run the flow against the live server
$ pytest -q; run the flow against the live server
56 passed; review 200, quote 200, enrolment row written

M0 to M3 are built and committed (latest 76f2c31). All 56 tests pass. I also ran the whole flow against the real server and Gemma. The phone run on your Pixel 7 is still to do, so M3 isn't fully done.

What now works
- read, check, batch, eval and serve, along with doctor.
- Quote math: checked by hand. For bill #1 the retailer lines come to $79.32 now against $64.0188 on the offer, a difference of $15.30, which is $16.07 with GST.
- Customer view: masks site IDs and hides flags.
- Negative savings: bill #8 shows "your current plan is cheaper" with the demo offers.
- Web flow over real HTTP: upload, review, confirm, the customer quote, and an enrolment row with no account number.
- Bug found and fixed: a bill that had already been read skipped creating its page images. There's a regression test.

Model eval (8 synthetic bills)
- Gemma 4 e4b: 5 of 8 quote-critical correct, all 8 valid, about 7 s a bill.
- Qwen3-VL 8b: 6 of 8, 2 outputs invalid, about 78 s a bill.

Show more
You

Re "Batch duplicate flag: batch doesn't run the duplicate-bill check (C16). The web app does." Fix this. Also draft the Dev post

Agent

Fixing C16 in batch first. I'll also show flag IDs rather than counts in the flags column, so the duplicate flag is visible.

▸ Bash edit cli.py; pytest -q
$ edit cli.py; pytest -q
58 passed
▸ Write POST.md
POST.md
File created
▸ Bash git commit -m "POST: correct claims about the phone photo and hardware"
$ git commit -m "POST: correct claims about the phone photo and hardware"
committed 1ccf427

batch now runs the duplicate check, and the post is drafted in POST.md. 58 tests pass and both changes are committed.

C16 in batch
- It flags a bill whose site and period match a different bill already processed.
- On the samples it now flags #3 and #4, which are the same bill as a PDF and as a phone photo.
- A bill never flags against its own folder.
- The flags column now lists flag IDs (like C01 C16) instead of counts. There are two new tests, with no model needed.

The post
- It follows the DEV template headings in order, with the intro line and the four tags.
- The hook, the "two lines" insight, how the checks work, the eval table, one failure story told straight, limits, why open, and the Author's Note are all written from what's in the repo.
- The failure story is the Gemma gas bill. It put the usage into the charge field, which would have told the customer they'd lose $117.70 when they'd actually have saved money. Check C01 caught it.
- Marked [TODO] in the post: the demo, the first-user timings, and the real phone photos.

Show more

The second is the last two days: real phone photos, the plausibility checks, the override, and the demo.

bill2quote, part 2: real phone photos, plausibility checks and the demo
You

Fix the site ID coding to match the SPEC. An electricity Site ID will not start 000. Regenerate the synthetic bills and related testing.

Agent

Electricity site IDs must now be 13 digits starting 00 (never 000), gas IDs 13 digits starting 000. I changed the regexes in config.py and config.example.toml, made the C07 message name the right prefix per commodity, regenerated the synthetic bills, and added a test for both prefixes. 60 tests pass.

▸ Edit/Bash config.py, checks.py, make_samples.py, tests
config.py, checks.py, make_samples.py, tests
60 passed