Same exchange, same period, same question. Three answers, and only the last one is about the market.
I wanted to know how many perpetual contracts my backtest universe should contain. It is a mundane question with a mundane answer, and it took me three attempts.
98, then 101, then 157.
v0: ask the exchange — 98
The obvious method. Call the exchange's listings endpoint, take every contract currently trading, done.
This gives you exactly the set of things that survived to today. Everything delisted is simply absent — not filed under "delisted", just not there. The API is not lying; it is answering "what is listed now", which is a different question from "what existed during my sample".
A universe of 98 survivors, tested over a period when 59 more contracts came and went, will show you a result that the market never offered. You could only have traded the ones that made it.
That is textbook survivorship bias. It is also easy to believe you are immune, because you did not filter anything — you just used the exchange's list. The bias was upstream, in the question I asked.
v1: add delisted contracts — 101
So I pulled the full universe including delisted contracts: 157 total. Then I filtered to those with meaningful trading volume, so I would not simulate coins nobody traded.
The filter was: "has at least X volume over the last 90 days of available data."
Result: 101 contracts. Barely different from 98. Almost every delisted contract got filtered out.
Why? Because a delisted contract's "last 90 days of available data" is the 90 days ending at its delisting — when trading volume collapses to near zero. My filter was asking "was this liquid recently", and for a dead contract the answer is always no. My own quality filter was re-introducing the survivorship bias I had just fixed.
This is the part worth internalising. I did the right thing (include delisted contracts) and then undid it with a reasonable-looking filter. The bug was not in the data, it was in the window I measured over.
v2: measure before the end — 157
The fix is a single change: for delisted contracts, measure volume over the 90 days before delisting, not the 90 days ending at the last available row.
That is the window in which the contract was actually being traded by people. And with that window, the answer became 157 — the full universe.
Three numbers, one question:
| version | what it did | contracts |
|---|---|---|
| v0 | exchange's current listing | 98 |
| v1 | full universe, filtered by volume in the last 90 days | 101 |
| v2 | full universe, filtered by volume 90 days before delisting | 157 |
Why 59 contracts is a big deal
The missing 59 are, by construction, the ones that failed. A survivorship-biased backtest over this universe is not slightly optimistic — it is testing a market in which you could only have picked winners.
The size of the effect depends on your strategy. A cross-sectional momentum strategy that picks the strongest contract each period will be dramatically distorted, because the losers were never candidates. A single-instrument timing strategy is less affected, though not unaffected: the instrument you chose is a survivor.
The transferable lesson
"I did not filter anything, I just used the exchange's list" is still a filter. It is the filter "must still be alive today", applied by someone else, and it is invisible because you never wrote it.
And the second-order lesson is the one that bit me: a reasonable quality filter can undo a correct universe correction, silently. If you include delisted instruments and then require recent liquidity, you have re-imposed survival without noticing.
Two rules I use now:
- State the universe as a sentence with a date range — "all perpetual contracts that traded at any point between 2020-01 and 2024-12, including those delisted before 2024-12". If you cannot write that sentence, you do not know what you tested.
- Measure every per-instrument filter over each instrument's own active window, not a fixed calendar window ending at the end of your data. The end of the data is a date that only survivors share.
A footnote on the volume number itself
Even with the right universe, I had to accept that the "volume" field I was filtering on was not what it appeared to be — in one of my data sources it is the number of ticks, not traded size.
That is a separate article. But the pattern is the same one: the column name told me what the field was called, not what it measured. Both this and the survivorship story are about the difference between a dataset's interface and its contents.
Universe construction is outside what a trade-log linter can check — it only sees the trades you gave it. It can tell you the sample size you need for a given edge, which is usually the moment you realise how much a missing 59 contracts matters.
There is also a *$0** download on Gumroad that bundles the tool with the demo
logs, if you want something to test against before pointing it at your own data —
name your price, and $0 is a valid answer: https://weiwave23.gumroad.com/l/backtest-lie-detector*
