A price comparator looks like a simple product: show a product, show a list of prices, redirect to a shop. I built one alone, and the simple part turned out to be about ten percent of the work. This post walks through the architecture of Runometry, a comparator for running shoes, bikes and sports electronics, and what I would tell myself before starting.
The numbers, so you know the scale: more than 600 000 prices, more than 20 shops, 15 delivery countries, a site in 10 languages and more than 2 500 products in the catalog. It is small by marketplace standards, but big enough that every shortcut shows up as a bug.
The core idea: static-first
Most comparator pages do not change every minute. A shoe's spec sheet changes when a human edits it. Prices change hourly at best. So I split the problem in two:
- Slow data (models, brands, specs, texts, taxonomy) lives in the repository as JSON and is compiled into pages at build time.
- Fast data (prices, availability) lives in a database and is served through a small API that the page reads after it loads.
The front end is AnalogJS, which is Angular with Vite, file-based routing and SSR plus prerendering. Pages are prerendered to HTML, so a crawler or a slow phone gets real content immediately, and Angular hydrates on top. Anything that needs the browser (a price list that depends on the visitor's country, a size filter) runs after the first render, and the template for it is deferred.
The rule I enforce in code review, even though I am the only reviewer: nothing that touches the browser runs during server render, and nothing blocks a crawler. It sounds obvious. It is the rule I broke most often early on.
The platform: Workers, D1, R2, KV
Everything runs on Cloudflare, and I picked it mostly for the boring reasons: one bill, one deploy tool, and edge latency for free.
- Workers serve the redirect that sends a visitor to a shop, the prices API, scheduled jobs and a small admin.
- D1 (SQLite) stores prices and offers. The schema is plain tables with indexes shaped by the queries the pages actually run.
- R2 stores images and pre-rendered pages.
- KV holds small, read-heavy things that are annoying to query: configuration snapshots and caches.
D1 taught me the first real lesson. SQLite at the edge is fast for reads and unforgiving about heavy writes. Early on, an ingest run that wrote rows one by one overloaded the database and the API started returning errors for visitors. The fix was boring: batch the writes, skip rows that have not changed, and keep a clear separation between the tables that visitors read and the tables the ingest writes while it works.
The hourly ingest Worker
Prices come from retailer feeds, which are files published through affiliate networks. A scheduled Worker runs every hour. Per feed, it does roughly this:
// Illustrative pipeline, not the real code const rows = await fetchFeed(account); const offers = []; for (const row of rows) { const match = matchToCatalog(row); // model, colourway, size if (!match) { report.unmatched.push(row); continue; } offers.push(toOffer(match, row)); // price, stock, link } await upsertChanged(offers); // only rows that differ await retireMissing(account, offers); // gone from the feed = gone from the site Three details mattered more than the code shape:
- Dry runs are the default. Running ingest on my machine never writes to production. Writing happens only at release, through the pipeline.
- Unmatched rows are a first-class output. A report of what did not match is more useful than a match rate. It tells me which model is missing from the catalog, or which mapping is wrong.
- Retire what disappears. A comparator that keeps showing a price the shop removed yesterday is worse than one with fewer prices.
The hard part: matching feed rows to catalog models
This is where the time went. A retailer feed row is a product title, a price, a stock flag, sometimes a size, sometimes an EAN, and a free-form description. My catalog has a canonical model, with colourways, and sizes beneath those.
The matching has to answer three questions:
Which model is this? Titles lie. The same shoe appears as "Brand Model 14", "Brand Model v14 Men's" and "Model 14 Running Shoe Black". Brands reuse names across versions, so "Model 14" and "Model 15" must never merge, while "Model" in a title alone matches too many things. I ended up with a layered approach: exact identifier (EAN) first when the feed has one, then a normalised brand plus model plus version match, then fuzzy rules that only ever produce a candidate for review, never an automatic match.
Which colourway? Feeds describe colour in marketing names, in codes, or not at all. I treat colour as a filter on a model page, not as a page of its own. That decision removed a whole class of duplicate-content problems and kept the number of URLs sane.
Which size? Sizes are the quiet killer. A pair of shoes is priced per size, and shops list sizes in EU, UK, US or centimetres, with half sizes written four different ways. If you compare the cheapest price without knowing the size is in stock, you are showing a price nobody can use. So the offer is per size, and the visitor filters by their size. Bikes added another version of the same problem: frame sizes are letters in one feed and numbers in another.
EAN looks like the clean answer, and it helps a lot, but it is not a silver bullet. Some feeds omit it, some share one EAN across sizes, and a few reuse them across colourways. I treat it as strong evidence, not as truth, and check it against the model it points to.
Offers follow the market, not the language
This is the design decision I am happiest about, and the one that is easiest to get wrong.
A site in German is not a site for Germany. A German-speaking visitor can be in Austria or Switzerland. A French site is read in Belgium, in Switzerland, and in overseas territories that belong to France. If a language decides which shops you see, someone in Zurich reading German gets prices from a shop that does not ship to Switzerland.
So I model two separate things:
- Locale: the language of the interface.
- Market: the country the goods are delivered to.
A locale lists the countries it can show, with a default. A page shows the offers delivered to the visitor's market when it is known and allowed by the locale, and falls back to the locale's default country otherwise. The language never filters offers. I also made one rule explicit and tested: a locale never silently falls back to another country's offers, because showing the wrong country's prices is worse than showing nothing.
The outbound link follows the language, though. If a shop has a site in the visitor's language, the click lands there; otherwise it lands on its storefront for that market.
Delivered price
A cheap shoe with expensive shipping is not cheap. Delivery rules live in data (per shop, per market: thresholds, flat fees, free delivery above a basket value), and the site can show a price with delivery included so you compare what you actually pay. By default prices are shown without delivery, and the delivered price is a toggle. Both are computed from the same data, so they never disagree.
The deal rule: a median, not a struck-through price
Every comparator is tempted by the same shortcut: take the shop's crossed-out "was" price and show a big discount. Those prices are often meaningless. A reference price can be set high and never actually charged.
I use a rule that depends only on my own observations. A price counts as a deal when it is at least 25 percent below the median of the daily lowest prices observed over the last 90 days. Two guards go with it:
- A discount above 70 percent is discarded. It is almost always a data error or a clearance of something that is not really what it claims to be.
- The shop's struck-through price is never used, in any form.
// Illustrative const median = medianOf(dailyLowest.slice(-90)); const isDeal = current <= median * 0.75 && current >= median * 0.30; The effect is that a "deal" on Runometry means cheaper than it has recently been across the market, not cheaper than a number a shop chose. It also means a brand-new model has no deal status until there is enough history, which is the correct behaviour.
A zod-validated catalog in git
All the slow data is JSON in the repository: shoes, bikes, electronics, brands, shops, delivery rules, translations, editorial text. Every file is validated with zod at build time, and a failing schema fails the build.
Why git instead of a CMS or a database?
- Review. A spec change is a diff. I can see exactly what changed and why.
- Provenance. Each sourced value carries a status (verified, reviewed, disputed) and a date. A verified value is never overwritten by an automatic pass.
- Rollback. A bad import is a revert.
- Speed. The build reads files. There is no query to run for a spec sheet.
The cost is that the repo grows, and tooling matters: scripts that normalise, validate and report are part of the product. I would not do it differently, but I would write the report tools earlier.
Lessons learned
- Solve matching before you scale shops. Adding a shop is easy. Adding a shop whose rows match correctly is the work. Count unmatched rows, not shops.
- Make the market a first-class concept on day one. Retrofitting "country" into a language-based design is painful.
- Keep production writes out of local development. A dry-run-by-default ingest saved me more than once.
- Static-first is a feature for SEO and for a solo dev. Fewer moving parts at request time means fewer pages at 3 a.m.
- Be honest in the data model. The deal rule and the delivered price exist because I did not want to show numbers I could not defend.
- Test the thing you changed, not everything. A full test run is slow. Targeted specs while working, the whole suite at release.
What is next
More product universes beyond shoes, bikes and electronics, more shops per country, and better matching for the long tail of models. If you have built something similar, I would like to hear how you handled sizes.
You can browse the full catalog to see the result of all this. Runometry is funded by partner links.
