Equity consumer research

Stock backtests: survivorship, universe changes and frozen rules

Investigate the historical candidate universe, data versions and decision chronology before comparing backtested equity results.

Source reference, reader evidence record and scoped conclusion
Preserve the source and the reader's observation separately.

Save the eligible universe for each decision

Today's stock list cannot automatically represent the securities that existed or qualified earlier. Keep additions, removals, identity changes and unavailable observations in a dated membership record. Ask whether delisted names and failed companies remain represented and how data gaps are handled. A consumer review need not reconstruct the entire market to ask this question, but it must not call a modern survivor list a verified historical universe.

Freeze rules before seeing the evaluation outcomes

Preserve filter settings, strategy code or rule text, source data version and the experiment boundary. Later improvements are new candidates, not an opportunity to rewrite the old test. For an entry-model comparison, define the causal entry objective first and assess exit economics separately. A favourable return after a revised exit cannot retroactively show which entry rule was ranked on untouched evidence. Our publication is not training or certifying a stock model.

Expose data treatment and execution assumptions

A backtest must specify raw or adjusted inputs, available company information, eligible sessions and hypothetical entry/exit rules. If a single bar touches both a stop and target, its range alone need not reveal the path. Retain that ambiguity unless appropriately granular evidence resolves it. A zero-cost simulation, a broker account statement and a copied account do not become comparable simply by formatting all three as percentages.

A demonstration is not a strategy validation sample

The fictional example illustrates how exclusion changes the sample rather than estimating how large survivorship bias is for an actual product. Software tests can confirm that a calculation matches stated assumptions; they do not supply independent strategy evidence. Request the preserved experiment and full evaluation population when a seller makes an out-of-sample claim. Missing files should remain a review limitation rather than be reconstructed from a persuasive chart.

Completed fictional case

Fictional historical universe has four equally weighted securities with evaluation returns +10%, +5%, -20% and -40%. Its stated equal-weight average is -11.25%. A present-day list that keeps only the first two produces +7.5%, an 18.75-percentage-point difference from omitting the other outcomes. This is pure declared arithmetic, not a measured market bias. The review conclusion is that the lists represent different populations, not that any named stock platform used the favourable list or committed deception.

RecordRepresented findingBoundary
Historical population(10 + 5 - 20 - 40) / 4 = -11.25%Four declared outcomes
Survivor-only copy(10 + 5) / 2 = 7.5%Different denominator
Difference18.75 percentage pointsNo real-provider finding

Primary references and inspection scope

Public-document inspection: 12 October 2026. The case arithmetic is original fiction, not a sourced provider outcome. No paid software, selected leader, broker route or subscriber account was tested.

Related equity questions