This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle
What I Built
I'm a freelancer in Paris. When I send an invoice, I don't know whether the client will pay on time. I also never know how to word the reminder: too soft and I keep waiting, too firm and I risk the relationship.
Late Pay Radar is a small app for freelancers that runs on your laptop:
- You add your invoices, as a CSV or as a text PDF exported from your invoicing tool.
- TabPFN, a tabular foundation model, ranks the unpaid ones by risk of late payment.
- You pick one. A local Gemma agent explains the risk and drafts a reminder in French or English: friendly before the due date, firmer after it.
Client names, amounts and payment history stay on the laptop.
The first user is me. I have one invoice on record: €1,725, issued on 30 June with 30-day terms, so due on 30 July.
Demo
The video (52 s, no sound) uses a fake invoice so no real client name appears on screen. A PDF is imported, its fields are extracted and confirmed, the risk table appears, and Gemma writes the explanation and the reminder. The wait in the middle (about 13 s) is local inference on my MacBook. At the end, ollama ps shows gemma4:e2b running on the GPU. My real invoice and what the app said about it are described further down.
There is no hosted version. Uploading your invoices to my server would defeat the point, so the app runs locally in 5 commands (see the README).
Code
Mialy333 / late-pay-radar
Predicts late invoice payments for freelancers, locally, with TabPFN + Gemma
late-pay-radar
Ranks your open invoices by late-payment risk and drafts the right reminder — entirely on your own machine (TabPFN + a local Gemma model via Ollama). Demo video: https://youtu.be/sccG2NXpBWw
Quick start
git clone https://github.com/Mialy333/late-pay-radar && cd late-pay-radar uv sync ollama pull gemma4:e2b uv run python scripts/download_barnet.py && uv run python scripts/build_barnet_dataset.py uv run streamlit run app.py Before that last command, you need a TABPFN_TOKEN in .env — see the next section.
TabPFN account (one-time)
The risk model (TabPFN) needs a one-time license acceptance before it will run locally, even for the demo data:
- Create a free account at https://ux.priorlabs.ai.
- Accept the TabPFN-3.5 license (non-commercial use) on the Licenses tab.
- Copy your API key from https://ux.priorlabs.ai/account.
-
cp .env.example .envand put the key in asTABPFN_TOKEN=<your key>.
This is a one-time step: once accepted, it's cached locally and no further account/network interaction is needed (see "Why local?"…
The code is MIT. The public dataset comes from the London Borough of Barnet under the Open Government Licence v3.0.
How I Built It
Stack: TabPFN-3.5 (local weights) for the risk score, Gemma (gemma4:e2b) through Ollama for the text, Strands Agents in Python for the agent and its tool, Streamlit for the interface, pdfplumber to read PDFs locally.
Finding real data
Freelancers don't publish their invoices, so I looked for public data. The London Borough of Barnet publishes late-payment reports: 1,561 supplier invoices with issue date, due date and actual payment date. Of the paid ones, 77% were paid more than 7 days late. These are suppliers waiting for a large client to pay them, which is the freelancer's situation.
I only used information available on the day an invoice is issued: amount, payment terms, the client's past behaviour, month and weekday. Barnet's dispute, hold and "paid within terms" columns describe what happened after the invoice was sent, so using them would have given the model the answer.
The backtest
I trained on older invoices and tested on the 446 most recent ones.
On the first run, TabPFN predicted "late" for almost everything. It recognised only 4 of the 123 invoices paid on time. On my synthetic test data it did the opposite and predicted "on time" every time. In both cases it was mostly repeating the most common answer.
The problem was my evaluation. I added a baseline that always predicts the most common answer, measured ranking quality with ROC-AUC (a freelancer mainly needs to know which invoice to chase first), and chose the decision threshold on a validation slice instead of the test set.
| Model | Balanced accuracy | ROC-AUC |
|---|---|---|
| Always the most common answer | 0.500 | n/a |
| Simple rule: "this client was late before" | 0.699 | 0.687 |
| TabPFN, no tuning | 0.598 | 0.688 |
TabPFN ranks invoices about as well as the simple rule, and makes worse yes/no decisions. It doesn't beat it.
Then I split the test set by how much history we have on the client:
| Test subset | TabPFN AUC | Simple rule AUC |
|---|---|---|
| New clients, fewer than 3 past invoices (186) | 0.629 | 0.584 |
| Known clients, 3 or more (260) | 0.766 | 0.779 |
The simple rule needs history. Without it, TabPFN can still use the amount and the payment terms. The gap is small and, on 186 invoices, not statistically solid. I still used it to design the product: new clients get the TabPFN score with a "⚠ low confidence" label, known clients get their own track record.
The agent
The Strands agent has one tool, get_invoice_risk. Gemma writes the explanation and the reminder. Most of the bugs I hit were fixed the same way, by moving the work that has one right answer out of the model and into code:
- Asked to find the riskiest invoices in a list of 78, Gemma picked from the middle. The ranking is now computed in code and shown as a table.
- It called an invoice that wasn't due yet "unpaid" and wrote a firm reminder. Due status, amounts and dates are now computed in code and passed to the model.
- It skipped the required note about where the training data comes from. The code now adds it every time.
- It sometimes left the amount or the date out of the reminder. A check now looks for both and asks for one rewrite. In testing, the first draft failed this check twice out of two, and the rewrite fixed both.
PDF import
My real invoice is a PDF, so in the morning I typed it into a CSV by hand. That's why PDF import was the first feature I asked for. The text is extracted locally, Gemma pulls out the fields as JSON, the code checks them, and you confirm them in an editable table before anything is scored. On my real PDF, Gemma read the due date correctly as 30 July. I was the one who had typed 31 July in my CSV.
What broke
My own invoice. Scored as if it had just been issued on 30 June, it got an 82% risk of late payment. My client paid 15 days early. TabPFN learned from UK public-sector suppliers, 77% of whom were paid late, and knew nothing about my client, my sector or my country. For a client with no history, the score mostly reflects that 77%. The app now shows this in a banner that the code adds, so the model can't leave it out: "Model trained on UK public-sector supplier invoices; transfer to other clients is uncertain."
A safety check that let a wrong answer through. To cut the 13-second wait, I merged the explanation and the reminder into a single Gemma call. Latency dropped to 4 seconds, and the log said Tools called: none. Gemma never looked up the invoice. It wrote "$1,500.00" and "October 15, 2026" instead of €3,250.00 and 10 May 2026. My check didn't catch it because it only ran when the tool had been called. I went back to two calls and changed the check: the app now looks up the invoice itself, always compares the reminder against it, and shows no reminder if the comparison fails.
My README. I cloned the repo into an empty folder and followed only the README. It crashed. The raw Barnet files were excluded from git, uv sync and ollama pull weren't mentioned, and the TabPFN token was explained at the bottom although the page needs it to load. Barnet's server also refused the download from Python's default user agent. After the fixes, a fresh clone starts in 5 commands.
Why Does Open Innovation Matter?
- Client data stays on the laptop. TabPFN's weights are cached locally. I tested it with all network traffic blocked: scoring still works, same result, 0.29 s. I also turned off Streamlit's usage statistics.
- No cost per use. There is no per-token bill. A tool priced at a few euros a month can't absorb API costs on every reminder.
- I could check the claims myself. The packages are open, so I could read the code and test that nothing is sent over the network instead of relying on a privacy policy.
- The parts can be swapped. Gemma, the agent framework and the scoring model are independent. The simple rule came close to TabPFN, so replacing the scorer is a real option.
One limit matters for a side hustle: TabPFN-3.5's weights are under a non-commercial licence. That's fine for this prototype. Selling the tool would require Prior Labs' commercial licence or a different scorer, and I'd rather know that before the first paying customer.
The first user
I like how warm the reminders are. The 82% left me with a question I'm still asking: how can anyone really measure that, and through what lens? Still, it gives me an estimate of the risk. It helps me organise myself, saves me time and, when needed, helps me find common ground with a client.
Part of the answer: 82% means that among the Barnet invoices that look like mine (30-day terms, similar amount, no client history), about 8 in 10 were paid late. It describes someone else's data, not my client. A single invoice also can't confirm or contradict a probability. That takes many invoices: of all those scored around 80%, about 80% should end up late. So the next step is to learn from my own payment history.
What I asked for as a user: PDF import and warmer reminders that explain why paying on time helps and offer to discuss a schedule. Both were built this weekend. Next: photos and scanned invoices, and scores based on my own history instead of Barnet's.
Would I pay for it? Yes, if it's secure and the automation saves me real work. My ceiling is €19.99 a month. At that price, a local model with no per-use cost is what makes the numbers work, once the TabPFN licence question is settled.
My Agent Session
Built with Claude Code over three saved sessions: the backtest, the agent, and the interface with the safety-check bug.
Build a one-page web interface for the invoice risk tool: pick a data source (a public demo dataset or your own invoices), see a ranked table, then pick one invoice and a language to get an explanation and a reminder from the local agent.
Built it as a single-page app. The ranking table is computed entirely in plain code, not by the language model, so it's deterministic and instant. The agent is only called once a specific invoice is selected, to explain its risk and draft a reminder. Also checked, before claiming the tool runs entirely locally, whether the underlying risk model or the web framework quietly phone home -- confirmed neither does once set up correctly, and disabled the one anonymous usage-telemetry setting the web framework has by default.
Let people also import their own invoices from a PDF, not just a spreadsheet. Extract the text locally, reject scanned image-only PDFs with a clear message, have the local model pull out the structured fields as data, validate them, and let the person review and correct everything in an editable table before anything gets scored. Nothing should ever be written to disk.
Built that pipeline: local text extraction first (scanned PDFs are caught and rejected immediately, before even trying the model), then the local model is asked to return the fields as structured data, which is checked in code (valid dates, positive amount, sane date ordering) with one retry if something's off and only the still-broken fields blanked out -- never the whole record. Everything stays in memory; confirmed by generating and testing with a fake sample invoice, never anything real.
Fix how the reminder reads: it should come from the freelancer who sent the invoice, to the client who owes the money -- first person, mentioning the invoice and amount and ending with a signature line. The explanation should also be in plain everyday words in whichever language was requested, not technical field names. And make sure the displayed amount is formatted the way each language actually writes money, not just by currency.
Fixed the perspective (the explanation was accidentally implying the user received the invoice rather than sent it -- corrected in both languages). Explanations now read in plain language, in whichever language was asked for. The displayed amount now formats itself to match the reply's language rather than always defaulting to one style, and the validator that checks a drafted reminder for the real figures was updated to recognize either language's way of writing the amount and the date, since the small local model doesn't always pick the one you'd expect.
Make the reminders warmer and more human -- greeting, a plain-language reason timely payment matters, an offer to help or discuss a schedule if timing is tough, and only mention a possible penalty when the invoice is genuinely overdue, never inventing a specific number. Then separately: fix the interface so switching the language doesn't lose or re-trigger anything, and make sure repeated interactions stay fast.
Updated the reminder style and added a line-by-line check: while testing I found the small model doesn't always write an English date the way the French one does, so loosened the check to accept either common ordering rather than forcing one. For the interface, grouped the invoice/language/button controls so changing the language alone no longer reruns anything until the button is actually pressed, cached the underlying model connection and the computed table so repeat interactions are instant, and made results stick around per invoice-and-language combination instead of disappearing on unrelated screen updates. Also deliberately tried to reproduce the exact scenario the user was worried about (switching language right after confirming an import) using an automated interface-testing tool rather than guessing, and found no error either before or after the change.
Reorder the reminder so it leads with the concrete facts (invoice number, amount, due date) right after the greeting, before the explanation of why it matters. Separately, try combining the two model calls (explanation and reminder) into a single call to cut latency.
Reordered the reminder as asked. The single-call experiment did cut latency roughly in half when it worked -- but it made the small local model noticeably less reliable: in some runs it skipped actually looking up the real invoice data at all and just invented numbers in the reminder, and because the safety check only ran when the lookup had actually happened, a run where the model skipped it sailed through uncaught. That's a guardrail that fails OPEN -- it protects you only when things already went right, which defeats the point.
Revert to the two-call approach. And fix the guardrail properly: fetch the real invoice data in code BEFORE calling the model at all, and always check the drafted reminder against that real data, whether or not the model's own lookup happened. If it still doesn't check out after one retry, don't show it -- show a clear error instead. Same for a visibly broken explanation.
Reverted to two separate calls. The real fix: the invoice's true numbers are now fetched directly in code, independent of anything the model does, and the reminder is always compared against those real numbers -- a run where the model never looks anything up now correctly fails the check instead of slipping through. If a reminder still doesn't check out after one retry, or an explanation comes back obviously broken (too short, or shouting in capitals), neither is shown -- a clear error is shown instead. Tested three times with a fake invoice: every reminder now correctly contains the real amount and date. One remaining honest finding: the explanation text itself sometimes came back as a flat 'I couldn't find that information' even though the real lookup clearly happened (the reminder had the right numbers) -- a model reliability quirk the length/formatting check doesn't catch, since the refusal is itself a normal-looking sentence. Flagged rather than hidden.
Prize Categories
- TabPFN (Prior Labs): the risk scorer, backtested against baselines on real invoices, with separate handling for new clients.
- Gemma: explanations, reminders and PDF field extraction, all running locally through Ollama.
