Building a Merchant-to-Category Learning Pipeline That Runs Entirely on Your Phone

When you review your monthly spending, you do not want to see a raw list of bank reference codes like UPI/CR/429810291/SWG*BLR.

You want to see:

  • Food & Dining: Rs. 12,450
  • Groceries: Rs. 8,200
  • Fuel: Rs. 4,100
  • Utilities: Rs. 3,500

To make transaction tracking effortless, an expense tracker must automatically map messy merchant strings into human-readable categories.

Commercial apps solve this by harvesting merchant data in the cloud. Every time millions of users swipe their cards, the app uploads the merchant strings to central servers, where crowdsourced tags categorize the business.

The privacy cost is immense: central servers track which coffee shops you frequent, which pharmacies you visit, and which fitness centers you attend.

Here is how FinLedger architected an on-device merchant-to-category learning pipeline (LocalCategoryIntelligenceEngine.kt, commit e4b4df2) that learns your personal categorization preferences entirely offline.

The Merchant Normalization Challenge

Before a system can categorize a merchant, it must normalize the messy strings generated by payment processors.

Consider how the same merchant appears across different banks and payment channels:

  • "SWIGGY BANGALORE IND"
  • "UPI-SWIGGY-swiggy.payu@icici-429182"
  • "POS 491029 SWIGGY FOODS"
  • "BUNDL TECHNOLOGIES PVT LTD" 
flowchart TD RawStrings["Raw Bank SMS Merchant Strings: 'SWIGGY BANGALORE IND' 'UPI-SWIGGY-swiggy.payu@icici' 'POS 491029 SWIGGY FOODS'"] --> Normalizer["Deterministic String Normalizer: 1. Strip payment rail prefixes (UPI-, POS, NEFT) 2. Strip VPA handles (@icici, @okhdfc) 3. Strip terminal IDs and location suffixes"] Normalizer --> CleanKey["Normalized Merchant Key: 'SWIGGY'"] CleanKey --> Pipeline["Local Category Engine"]

FinLedger's normalizer strips transaction prefixes (POS, UPI, IMPS, ACH), removes VPA routing handles (@oksbi, .payu@icici), and discards dynamic terminal codes.

This extracts a clean, stable merchant root key—such as SWIGGY, ZOMATO, SHELL, or UBER—in less than 20 microseconds.

The Multi-Tiered On-Device Pipeline

Once the merchant key is normalized, the engine evaluates categorization through a prioritized three-tier hierarchy:

flowchart TD CleanKey["Normalized Merchant Key"] --> Tier1{"Tier 1: User Personal Overrides (Learned Local Relational Map)"} Tier1 -->|Found Exact Match| Result1["Apply User Learned Category (Confidence: 100%)"] Tier1 -->|Not Found| Tier2{"Tier 2: Pure Kotlin ML Model (TF-IDF + Logistic Regression)"} Tier2 -->|Confidence > 75%| Result2["Apply ML Inferred Category (Confidence: High)"] Tier2 -->|Uncertain| Tier3["Tier 3: Default Fallback (Flag for Review / Other)"]

Tier 1: The Local Relational Feedback Loop

The user is always the ultimate authority. If a user buys office supplies at a department store and categorizes it under Work & Business, that preference belongs to them, regardless of what a general dictionary says.

In FinLedger, whenever a user edits or confirms a transaction's category:

  1. The app updates an internal Room SQLite entity: merchant_category_preferences.
  2. The record maps the normalized merchant key (normalized_merchant) to the selected category ID (category_id), along with an incremented usage counter and last-updated timestamp.
  3. The next time an SMS arrives with that merchant key, Tier 1 intercepts it immediately. The app applies the user's custom choice with 100% confidence.

Tier 2: The Lightweight Inference Engine

For merchants the user has never encountered before, Tier 2 evaluates the normalized string and surrounding SMS context using FinLedger's pure Kotlin machine learning engine (LocalSmsIntelligenceEngine.kt).

Using sublinear TF-IDF and logistic regression, the engine detects semantic markers:

  • Keywords like "Fuel", "Petrol", "HPCL", or "IOCL" yield Transportation & Fuel.
  • Keywords like "Hospital", "Pharmacy", "Clinic", or "Medicals" yield Healthcare.
  • Keywords like "Supermarket", "Grocery", or "Mart" yield Groceries.

If the top predicted category has a calibrated Softmax probability exceeding 75%, it is assigned as the default suggestion.

Tier 3: The "Never Guess" Fallback

If the merchant is novel and the ML model is uncertain (e.g., an unfamiliar regional shop name like "Guptha & Sons"), the pipeline refuses to guess.

The transaction is assigned to General / Uncategorized and highlighted in the pending review tray with a prompt inviting the user to pick a category. The moment the user picks a category, Tier 1 learns it permanently for all future visits.

Memory and Storage Footprint

Because everything runs on SQLite and in-memory caches:

  • The local merchant dictionary occupies less than 200 KB of encrypted SQLite storage for a typical user over two years.
  • Lookup queries execute against indexed columns in less than 0.3 milliseconds.
  • No background network daemons exist; the database never syncs merchant names to external cloud servers.

Key Takeaway

A smart personal finance app does not need to send your shopping habits to the cloud to understand where you spend your money.

By combining deterministic string normalization, a local relational feedback loop, and a lightweight on-device ML engine, FinLedger creates a categorization pipeline that gets smarter with every transaction you log—without sacrificing your privacy.

FinLedger is a privacy-first, offline-first personal finance application for Android that models bank accounts, credit cards, and cash without double-counting.

Explore FinLedger on Google Play: https://play.google.com/store/apps/details?id=com.devanshufinance.app