I built a private bookkeeper you can talk to — and learned where a small local model actually earns its keep

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle.

Every freelancer knows the tax-time ritual: scroll a year of bank lines, tag each
one, and try to remember whether SQ *BLUE BOTTLE 4471 was a client meeting or a
personal coffee. The obvious "AI solution" is to upload your whole financial life
to a SaaS and let their model sort it. I wanted the opposite: do it on your own
laptop, offline, and let the model do the part that only a language model can do.

So I built Deductible — a local-first bookkeeping copilot. It categorizes your
transactions with an open-weight LLM running entirely on your machine, hands you a
Schedule-C-ready CSV, and then lets you ask your books questions in plain English —
with no data ever leaving the laptop.

Along the way I benchmarked three model sizes honestly, and the results changed
what the product is. This post is about that.

What I Built

Deductible does two jobs:

1. Categorize a bank/card CSV into 10 expense categories, apply the right tax treatment (meals 50%, personal 0%, the rest 100%), compute totals, and flag items worth a human glance. Output is an auditable CSV — the numbers come from plain Python, not from a model's vibes.
2. Explain. ask.py builds a deterministic snapshot of your finances and lets the local model answer questions like "What are my biggest deductible expenses, and what should I double-check before filing?" — offline.

The whole thing runs on an open-weight model (Qwen2.5 via llama-cpp-python,
GGUF) with no network access, no API key, and no per-row cost.

Demo

Categorize, then ask:

python deductible.py --csv sample_data/transactions.csv --out taxes_ready.csv python ask.py --csv taxes_ready.csv \ --question "What are my biggest deductible expenses, and what should I double-check before filing?" 

The ask layer prints a snapshot and then the model's answer:

Headline facts (precomputed): - Total estimated deductible: $3,488.11 - Deductible category #1: Equipment = $1,999.00 - Deductible category #2: Professional Services = $482.00 - Deductible category #3: Travel = $371.60 Flagged for review (possible misclassification or large amount): - MACBOOK PRO 14 M4: $1,999.00 -> Equipment - NETFLIX SUBSCRIPTION: $15.49 -> Other/Personal ANSWER -------- The biggest deductible expenses are for Equipment ($1,999.00) and Professional Services ($482.00). You should double-check the classification of the MACBOOK PRO 14 M4 and the NETFLIX SUBSCRIPTION to ensure they are correctly categorized as equipment and personal expenses, respectively. 

Note the design choice: the model never does the math. The snapshot is
precomputed by auditable Python; the model only explains and advises. That is the
difference between a demo and something you'd trust with your taxes.

Code

GitHub: https://github.com/musafirilmu21-max/deductible

• deductible.py — categorizer + CSV export
• ask.py — the "talk to your books" layer
• benchmark.py — the accuracy table below
• test_deductible.py — tests for the deterministic parts
• Sample data and a clean MIT license.

How I Built It

The interesting part is what the benchmark forced me to change.

I started with the obvious design: the model classifies each transaction. To make
a tiny model reliable I constrained its output to a single number 1–10 and gave it
one few-shot example per category. Then I measured. I labelled two small datasets —
clean merchant names, and messy real bank strings like
SQ *BLUE BOTTLE COFFEE 4471 SF — and ran three open models of increasing size:

Two uncomfortable truths fell out:

1. A hand-tuned keyword table beats the models on clean names. Of course it does — I wrote the keywords against those exact names. On real statement garbage everything drops, and the small models did not magically beat rules.
2. Bigger was not automatically better in this task — 3B did not beat 1.5B.

If I stopped there, the honest conclusion would be "don't use a model for this."
So I asked the better question: what is a language model actually good at here?
Not memorizing that "Starbucks = meals" — a rule does that for free. It is good at
talking to a human about their money. Turning a spreadsheet into a bookkeeper
you can ask "what should I double-check before filing?" — that no keyword table
can do.

So I kept the deterministic classifier as the reliable spine (with a keyword net as
a safety fallback, and a --dry-run mode that needs no model at all), and made the
natural-language layer the star. The numbers stay in Python; the model does
language. That split is the whole architecture.

Why Does Open Innovation Matter?

• Privacy is the product. Financial data is the most sensitive data a freelancer owns. Running the model locally means it never leaves the laptop — no ToS, no breach surface, no vendor seeing your ledger. A closed API can't offer that at any price.
• $0 marginal cost. Tax season is bursty. An API billed per token turns a free tool into a bill you avoid using in March. A local model costs nothing per run.
• Swappability as a feature, not a limitation. Open weights let you choose the brain: a 0.5B for a Raspberry Pi, a 1.5B for a laptop, or a fine-tune on your own ledger for better accuracy — by changing a file path. My benchmark above only exists because weights are open and downloadable. You can't A/B three models behind one closed endpoint.
• Honest tooling. Because I can read and swap the model, I can benchmark it and tell you the truth about where it's weak. That transparency is only possible in the open.

My First User

Not yet — this was built in a day. The explicit next step is to hand it to a
freelancer for one real quarter of transactions and report back what they used,
what they ignored, and what they asked for next. (Early signal from showing the
ask.py flow: people want a way to teach it their recurring vendors — which is
exactly the fine-tune path open weights make possible.)

Prize Categories

No partner-specific technology used — built entirely on open-weight Qwen models via
llama-cpp-python. Entering for the overall prize and the Best Use of an
open-weight model spirit of the prompt.

If you got value from this, try benchmark.py on your own bank export and tell me
how your vendors do — I'd genuinely like to see where the small model breaks.