Let the Model Read, Let the Code Convert: Building a Bill OCR for Nepal

How I turned photos of Nepali paper bills into reviewable, exportable data with Next.js, Gemini and Prisma 8.

The problem

In Nepal, many small businesses still receive printed or handwritten bills. An accountant then types each one into Excel or accounting software: vendor, PAN number, date, items, VAT, total. It is slow, and one wrong digit in a total or a PAN can cause mismatches at tax filing time.

I wanted to build something people would actually use, so I picked this problem: take a photo of a bill, get structured data back, let a human check it, and export it. The result is Bill OCR

Generic OCR tools don't fit well because Nepali bills mix Devanagari numerals (०१२३) with English digits, use Bikram Sambat (BS) dates like 2083/07/22, carry 9-digit PAN/VAT numbers, and come in every layout imaginable.

The one design rule

The model reads. The code decides.

A vision model is excellent at looking at a messy photo and finding the vendor, the date and the total. It is much less trustworthy at arithmetic or calendar conversion, and you can't unit test it. So the model's only job is to copy what is printed. Everything after that (converting numerals, parsing amounts, validating PAN numbers, converting BS dates) is ordinary code with tests.

The pipeline

Upload → validate (type, size, file signature) → store the image → extract with Gemini (JSON, values exactly as printed) → validate the raw output with Zod → normalize in code and collect warnings → save as PENDING_REVIEW → human review → CONFIRMED → CSV export 

Step 1: Extraction

The prompt asks for JSON only, with fixed keys, and sets rules that matter more than the wording of the task:

• Copy values exactly as printed. Don't convert numerals or dates.
• If a field is missing or unreadable, return null. Never guess.
• Keep item names in their original language.

"Never guess" is the important one. A null shows up in the review screen as an empty field. A confident wrong guess looks like a correct value.

I validate the response with Zod, where every field is nullable, so one unreadable field doesn't throw away the whole bill:

export const RawBillSchema = z.object({ vendor_name: z.string().nullable(), vendor_pan: z.string().nullable(), bill_number: z.string().nullable(), bill_date: z.string().nullable(), subtotal: z.string().nullable(), vat: z.string().nullable(), total: z.string().nullable(), items: z.array(z.object({ name: z.string(), qty: z.string().nullable(), rate: z.string().nullable(), amount: z.string().nullable(), })), }); 

Note that every value is a string, exactly as printed. Numbers come later.

I built this on Gemini's free tier, which sometimes answers 503 "model is overloaded". So extractBill retries with short waits and then falls through a list of models read from an environment variable. A request timeout and a fallback model stop a busy model from hanging the upload for over a minute (I saw 87 seconds once).

Step 2: Normalization

Now the deterministic part. Nepali numerals are a one-line map:

const DIGITS: Record<string, string> = { '०': '0', '१': '1', '२': '2', '३': '3', '४': '4', '५': '5', '६': '6', '७': '7', '८': '8', '९': '9', }; export const toEnglishDigits = (s: string) => s.replace(/[०-९]/g, (d) => DIGITS[d]); 

Amounts like रु. ४,८००/- become 4800 after converting the digits and stripping currency markers and commas. Anything that doesn't parse as a number becomes null rather than NaN. PAN/VAT numbers must be exactly 9 digits or they are rejected.

Dates were the most interesting part. A bill may carry a BS date (2083-07-22) or an AD date, and I need both stored. I convert with nepali-date-converter, then validate by converting back:

const ad = new NepaliDate(year, month - 1, day).toJsDate(); // Round trip: if converting back gives a different BS date, // the input was invalid (for example day 32 in a 30-day month) if (new NepaliDate(ad).format('YYYY-MM-DD') !== bs) { return { bs: null, ad: null }; } 

Two bugs I caught in review are worth sharing because they are easy to miss:

1. Overlapping year ranges. I treated years 2000–2099 as BS and 1900–2100 as AD. An AD date like 2026-11-08 matched the BS branch and was read as BS year 2026, which is around 1969 AD. The ranges must not overlap. In practice bills are either around 2060 and later (BS) or 1990–2059 (AD).
2. Time zones. One branch produced a date at local midnight and the other at UTC midnight, so an export could show some dates a day early depending on the server's time zone. Store the calendar day as UTC midnight everywhere.

Step 3: Catching mistakes

Even a good extraction misreads digits sometimes. Bills carry their own redundancy, so I use it:

• subtotal + VAT should equal the total
• the item amounts should add up to the subtotal
• quantity × rate should equal an item's amount

A mismatch doesn't block anything. It becomes a warning on the review screen, because a bill's own printed numbers are sometimes wrong and the accountant should decide. Hard errors (no vendor, no total, malformed PAN or date) block confirming.

The review screen shows the original image next to an editable form. The same validation function runs in the browser for live feedback and again on the server at confirm time, so the server never trusts the client.

Step 4: Accounts and authorization

A bill contains PAN numbers and amounts, so I treated access seriously:

• Better Auth handles email and password, GitHub and Google, with email verification and password reset by a 6-digit code, and optional authenticator-app 2FA.
• Every query is scoped by the signed-in user's id, and every API route checks the session itself. The Next.js proxy file only does a quick redirect for signed-out visitors, and I don't count it as a security boundary.
• Enabling the email-OTP plugin also adds an endpoint that signs you in with just an emailed code, which would skip both the password and the 2FA. I block that endpoint, so codes are only used to verify an email or reset a password.

One practical note: Prisma ORM 8 is new, and at the time of writing Better Auth's official Prisma adapter doesn't support it yet, so I used a community adapter and am watching for the official one.

What I haven't done yet

I haven't measured accuracy properly. The next step is collecting 20–30 real bills (printed, handwritten, crumpled, photographed in bad light), recording the correct answers, and reporting per-field accuracy, so every prompt change is measured instead of guessed. I also want unit tests for the normalization functions, especially BS/AD boundary dates.

If you copy this approach, check Google's current terms on how content sent to Gemini's free tier may be used before you send real client data.

Takeaways

• Let the model do what it is good at (reading messy images) and keep conversion, validation and arithmetic in code you can test.
• Prefer null over a guess, and show the nulls to a human.
• Use the redundancy already in the document, like totals, to flag likely misreads.
• Treat the human review step as part of the product, not an afterthought.

If you handle Nepali invoices or build document pipelines, I'd love your feedback: Bill OCR