Bolo turns speech on a phone into text. In cloud mode the phone uploads a voice note to a small Node.js proxy, and the proxy asks a speech-to-text backend for the transcript. There is more than one backend. Most requests go to a cheaper primary path; when that path is slow or down, a metered fallback answers instead, and the fallback costs many times more per minute of audio.

That design is good for users, because a dictation almost always comes back. It is bad for the budget in one specific way: if the cheap path breaks quietly, everything lands on the expensive one, and nothing looks wrong. Every request still succeeds. The bill is the first alarm, and it arrives weeks late.

This post is about the small module that watches for that case and reacts in minutes, not weeks.

Disclosure: I'm the developer of Bolo, and this post is published from my account. It was written and published by an AI agent working on my behalf, from the project's source code.

Watch a ratio, not a total

A spend total is hard to alarm on. It grows with usage, so a busy day and a broken day look alike. The useful signal is the share of work the metered backend did over a trailing window. When the cheap path is healthy that share is small and steady; when it breaks, the share jumps towards 100% within a few requests.

So the guard keeps a list of completed transcriptions over the window, and for each one records two things: was it paid, and how many seconds of audio it billed.

record({ paid, seconds, nowMs = Date.now() }) { this.events.push({ at: nowMs, paid: Boolean(paid), seconds: Math.max(0, Number(seconds) || 0) }); this.evaluate(nowMs); }

The window defaults to 24 hours, with a floor of one minute so a bad config value cannot make it meaningless. Old events are pruned from the front of the array on every read, which works because they are appended in time order.

Two shares, either one trips it

Counting requests alone misses a failure mode: if only the long recordings fall through to the metered path, the request share stays low while the cost share is high. Counting seconds alone has the opposite blind spot. The guard computes both and trips if either is over the threshold:

const over = s.requests >= this.minRequests && (s.paidRequestPct > this.thresholdPct || s.paidSecondsPct > this.thresholdPct);

The default threshold is 15%, and it is a config value, so it can follow real measurements rather than a guess baked into code.

A minimum sample before any decision

minRequests (default 20) stops the guard from reacting to noise. Two requests, one of them paid, is 50%, and means nothing. Below the floor the guard always reads "not degraded".

That floor also defines what happens after a restart. The state lives in memory in one proxy process, so a restart starts with an empty window and reads as healthy until enough new requests have arrived. I wrote that down in the module's header comment, because it is the kind of behaviour that surprises the next person who reads a log after a deploy.

Unknown means paid

Each backend result carries a provider tag and, for the primary path, the name of the route that actually served it. The classifier is deliberately pessimistic:

function isPaidResult(result) { if (!result || typeof result !== 'object') return true; if (result.bolo_provider === 'supported_fallback') return true; if (result.bolo_provider === 'private_stt') { const path = String(result.path || ''); if (PAID_PRIVATE_PATH.test(path)) return true; return !FREE_PRIVATE_PATH.test(path); } return true; }

A missing result is paid. An unrecognised provider is paid. A route name that matches neither list is paid. The point is that adding a new backend can never make the service look cheaper than it is: someone has to classify the new route as cheap, on purpose, before it counts as cheap.

The opposite default is the one that bites. If unknown routes counted as cheap, a new metered route added during an incident would be invisible to the guard at exactly the moment it matters.

Degrade one tier, never the floor

When the guard trips, it does not shut anything off. Only the paid plan's daily allowance shrinks, and the shrink is clamped on both sides:

function proDailyLimitSeconds({ proDailySeconds, proDegradedSeconds, dailyTrialSeconds, degraded }) { if (!degraded) return proDailySeconds; return Math.min(proDailySeconds, Math.max(dailyTrialSeconds, proDegradedSeconds)); }

Two rules are encoded there:

  • A degraded paid user never gets less than a trial user would. A misconfigured proDegradedSeconds of zero cannot lock out people who paid.
  • The degraded value never raises the limit. If someone sets it above the normal allowance, the normal allowance wins.

Trial users are not touched at all. Their limit is already the smallest one.

Log transitions, not states

evaluate() runs on every request, but it only logs when the flag changes, and it logs the numbers that caused the change:

[cost-guard] DEGRADED paidRequests=9/40 (22.5%) paidSeconds=410/1500 (27.33%) threshold=15% window=24h

(Example numbers, not production data.)

One line per flip, at warn going into degraded and log coming out, means the log tells the story of an incident in two lines instead of burying it in thousands of identical "still degraded" lines. A snapshot() method returns the same counters plus the time of the last change, so an admin endpoint can show where the guard stands without anyone reading logs.

What it does not do

  • It is not a billing system. It never sees invoices or prices; it only knows which requests were paid.
  • It is per process. With several proxy replicas each would keep its own window, and a shared store would be the next step.
  • It reacts to a broken cheap path; it does not fix one. The degraded state is a brake while someone repairs the real problem.

Takeaways

  • When a fallback is much more expensive than the primary, alarm on the share of work the fallback does, not on total spend.
  • Measure the share by count and by size, and trip on either.
  • Require a minimum sample, and write down what a restart does to the window.
  • Classify unknown backends as expensive. Make "cheap" an explicit decision.
  • When you degrade, clamp the new limit between the floor you promised and the ceiling you already had.
  • Log the flip and its numbers once; expose a snapshot for everything else.

Bolo is on Google Play as "Bolo AI Voice Typing & PC Sync" (search "Bolo AI"): https://play.google.com/store/apps/details?id=com.ashwar.bolo&utm_source=devto&utm_medium=article&utm_campaign=2026-w46-costguard. Cloud dictation has a limited free cloud trial, and you can also bring your own OpenAI or Gemini key, or use offline on-device dictation.