One rebuild, not twenty: what caching a live market dashboard taught me

About one in four investors opening one of Fundai's market cards waited up to nine seconds for it. The numbers on it had barely changed. Our cache had simply been thrown away a moment earlier, and everyone arriving then paid to rebuild it.

Fundai is a stock-research platform for the Tehran Stock Exchange that I co-founded in 2023. Investors use it to look up companies and see the whole market at a glance: valuations, industries, returns. Some of those views sum up years of trading data for every listed company, so we cache them, which means keeping a ready-made copy so most visitors don't wait for the database at all.

Caching sounds simple. Keeping a cache fast, fresh and correct at the same time, while new market data keeps arriving during the trading day, turned out to be where the real work was.

The whole idea in one kitchen

Picture a busy restaurant.

Twenty people order the same dish at the same moment. A good kitchen has one cook make it and serves everyone from that pan; a bad one has twenty cooks make twenty identical dishes.
The specials board is being rewritten. A good restaurant leaves yesterday's board up until the new one is ready; a bad one shows a blank wall in the meantime.
Nobody hangs a board that's half written. Yesterday's complete board is better than today's with only the first three dishes on it.
When the menu changes, the new board gets a new number, so nobody has to go round hunting for old copies to take down.

Each of those was a real problem on Fundai, and each fix is that kitchen rule in code. The first three handle slow rebuilds and incomplete data; the fourth handles what happens when the data underneath changes.

What investors ran into

The market valuation page sometimes opened nearly empty. A data job writes each new day's ratios company by company over about ten minutes. The first visitor in that window got the market's average built from only the first part of the list, and that half-finished page was cached and shown to everyone for six hours.

The market returns card made people wait. Building it reads five years of every symbol's trading sessions, which took 1.5 to 8 seconds of database work in production. New price data arrives every few minutes during a session, and each update threw the card's cache away, so about one visitor in four waited 2.5 to 9 seconds.

And when a popular company's data changed, everyone who opened its page in that moment would hit the database with the same heavy query at once.

Deciding what the product should promise

Before changing code, I settled the trade-offs, because a cache is really a set of promises about what readers see.

A page must never show part of a day. An incomplete market average looks like a real number, and investors act on numbers, so yesterday's complete picture is better than today's half of one.

A card that's a few minutes behind beats a card that makes you wait nine seconds, as long as it's complete. During a trading session, the returns over five years barely move between two price updates.

Nobody waits forever. If the person "cooking" a page is slow or fails, other readers stop waiting after two seconds and get the data themselves, so a stuck build slows a page down rather than breaking it.

How it works

Every cached view is stored under a version number, one for each company and one for the whole market. When new data arrives, we don't search for old copies to delete; we move the version forward, and every reader starts looking under the new number. That's the new board with a new number. Deleting would have meant knowing every key a company has, one for each combination of filters a reader can pick, and a build already in progress could write its old result straight back after the delete. One increment retires every entry at once, and a late build lands under a version nobody reads any more.

On a miss, one reader takes a short lock and builds the page; the others wait up to two seconds for its result instead of building their own copy. Simplified, it looks like this:

data = cache.get(key) # key includes the version if data is None: token = random_token() if cache.add(key + ":lock", token, 30): # one reader wins the lock  try: data = build(); cache.set(key, data) finally: release(key + ":lock", token) # only if it's still ours  else: data = wait_for(key, seconds=2) or build() # bounded wait

The token matters more than it looks. My first version simply deleted the lock when the build finished, and a reviewer pointed out the gap: if a build ran past the lock's 30 seconds, the lock would expire, the next reader would take it, and the first reader's delete would then remove that reader's lock, letting a third start the same build. So each lock now holds a random token, and the release checks it and deletes it in one Lua step on the Redis server. If the build itself fails, that reader gets an error, the lock is still freed, and the next reader tries again.

The two-second limit trades a little duplicated work for bounded waiting: no reader is held up by a build that may have died, and a stuck lock frees itself after 30 seconds. It also means one build serves everyone only when the build takes under two seconds. A slower view would send the waiting readers off to build it themselves, which is why the slowest one, the returns card at 1.5 to 8 seconds, doesn't rely on this and serves its kept copy instead.

Go's singleflight, in the further reading below, does this inside one process. Fundai runs several web processes, so the lock lives in Redis, where every process sees it.

The returns card goes one step further. It keeps its last complete copy, stamped with the version it was built from. When a price update moves the version, the next visitor gets the kept copy immediately, and a single background job builds the new one. That's yesterday's board staying up while today's is written. The same Redis lock, with its own token, lets only one rebuild be queued at a time, and a rebuild that finishes after a newer one never replaces the newer copy. When no copy is kept at all, on the very first visit or after the copy has expired, one reader builds it and the others wait for that build. The review prompted me to test that path too: ten readers arriving at once ran ten builds before the fix, and one after.

For the half-written day, the job now marks a day complete only after its whole pass has finished, and every market view reads only up to the last complete day. Every price or ratio update also moves the market's version, so readers won't accidentally treat an older cached result as current. The returns card is the deliberate exception: it may serve an older complete copy, clearly labelled with its data date. And if the cache itself is down, every view is built straight from the database: slower, never broken.

How I checked it

A test sends twenty readers at a cold page at the same instant, with a build that takes 0.3 seconds. The build runs once, and all twenty get its result. With the lock removed, the same test builds the page twenty times. Other tests cover the lock being freed when a build fails, a build that outlived its lock leaving the next holder's lock alone, an older rebuild never replacing a newer kept copy, a reader whose "cook" never finishes building it themselves, a version change sending the next reader to rebuild, and the returns card answering from its kept copy while a rebuild is queued.

That leaves one question that matters for financial data: how old can the kept copy get? The card always shows the date its data runs to, so an older copy is labelled, not disguised. If rebuilds kept failing, the kept copy would expire after three days and the next visitor would build the card directly, and every failed rebuild is logged. Serving a slightly older but complete card was a deliberate choice; the returns over months and years shift only a little between two price updates, whereas a half-written day changes the market average completely.

What any product can take from this

Decide what readers may see before you decide what to cache: is a little stale acceptable, and is incomplete ever acceptable?
Never cache something that's still being written. Mark data complete first, and read only up to that mark.
When many people ask for the same expensive thing at once, let one build it and the rest wait a bounded time, and know what happens when the build takes longer than that wait.
Invalidate with a version number rather than by hunting down old copies.
If speed matters more than the last few minutes of freshness, serve the last good copy and rebuild in the background.