Equity consumer research
Stock backtests: survivorship, universe changes and frozen rules
Investigate the historical candidate universe, data versions and decision chronology before comparing backtested equity results.

Save the eligible universe for each decision
Today's stock list cannot automatically represent the securities that existed or qualified earlier. Keep additions, removals, identity changes and unavailable observations in a dated membership record. Ask whether delisted names and failed companies remain represented and how data gaps are handled. A consumer review need not reconstruct the entire market to ask this question, but it must not call a modern survivor list a verified historical universe.
Freeze rules before seeing the evaluation outcomes
Preserve filter settings, strategy code or rule text, source data version and the experiment boundary. Later improvements are new candidates, not an opportunity to rewrite the old test. For an entry-model comparison, define the causal entry objective first and assess exit economics separately. A favourable return after a revised exit cannot retroactively show which entry rule was ranked on untouched evidence. Our publication is not training or certifying a stock model.
Expose data treatment and execution assumptions
A backtest must specify raw or adjusted inputs, available company information, eligible sessions and hypothetical entry/exit rules. If a single bar touches both a stop and target, its range alone need not reveal the path. Retain that ambiguity unless appropriately granular evidence resolves it. A zero-cost simulation, a broker account statement and a copied account do not become comparable simply by formatting all three as percentages.
A demonstration is not a strategy validation sample
The fictional example illustrates how exclusion changes the sample rather than estimating how large survivorship bias is for an actual product. Software tests can confirm that a calculation matches stated assumptions; they do not supply independent strategy evidence. Request the preserved experiment and full evaluation population when a seller makes an out-of-sample claim. Missing files should remain a review limitation rather than be reconstructed from a persuasive chart.
Completed fictional case
Fictional historical universe has four equally weighted securities with evaluation returns +10%, +5%, -20% and -40%. Its stated equal-weight average is -11.25%. A present-day list that keeps only the first two produces +7.5%, an 18.75-percentage-point difference from omitting the other outcomes. This is pure declared arithmetic, not a measured market bias. The review conclusion is that the lists represent different populations, not that any named stock platform used the favourable list or committed deception.
| Record | Represented finding | Boundary |
|---|---|---|
| Historical population | (10 + 5 - 20 - 40) / 4 = -11.25% | Four declared outcomes |
| Survivor-only copy | (10 + 5) / 2 = 7.5% | Different denominator |
| Difference | 18.75 percentage points | No real-provider finding |
Primary references and inspection scope
Public-document inspection: 12 October 2026. The case arithmetic is original fiction, not a sourced provider outcome. No paid software, selected leader, broker route or subscriber account was tested.