How Many Trades Make a Backtest Trustworthy?

Written by chwoo · Crypto trader, 7 years. Running an automated futures bot for the past 4 months. · 2026-07-20

Here is an uncomfortable experiment: flip a fair coin ten times. Reasonably often you will get seven heads, or 70%. Nobody would conclude the coin is magic. Yet traders routinely see a strategy win 7 of its 10 backtested trades and conclude they found a 70%-win-rate system. Sample size is the least glamorous idea in backtesting and the one that quietly invalidates most of what you will ever test.

Why small samples produce spectacular results

With few trades, luck dominates skill. The fewer the trades, the wider the range of outcomes pure chance can produce, and because you only pay attention to the settings that happened to do well, small samples plus many attempts practically guarantee you will "discover" something amazing that is pure noise. This is the engine behind most too-good backtests: not bad math, just few trades and many tries.

A rough intuition for the error in a win-rate estimate: it shrinks with the square root of the trade count. Ten trades put roughly ±15 percentage points of fog around your measured win rate; a hundred trades still leave about ±5; only around several hundred does the picture get reasonably sharp. Exact numbers depend on the true rate, but the shape of the rule is what matters: to halve the noise you need four times the trades.

What this means in practice

Under ~30 trades: anecdote. Interesting, worthless as evidence either way.
~30–100 trades: a hint. Worth refining and re-testing, not worth funding.
Hundreds of trades, spread across years and regimes: the estimate starts to deserve trust, provided the strategy was not tuned to that same data.

This is why testing on seven years of data matters more than testing on the last six months, and why lower timeframes are not automatically worse: a 1h strategy can produce a thousand trades where a weekly one produces twenty. A weekly strategy is not untestable, but honesty requires admitting how wide its error bars are.

Sample size interacts with everything else

A large sample does not save you from overfitting if you tried hundreds of parameter combinations. Each try gives chance another lottery ticket. And a large sample concentrated in one bull market is really one long correlated bet, not hundreds of independent ones. The period table on this site exists precisely for this: performance split across 7y / 3y / 1y windows shows whether trades from different eras agree with each other.

Before believing any backtest, yours or anyone's, ask two questions: how many trades, and across how many different market conditions? If the answer is "forty trades, all from 2024," you are looking at a story, not a statistic. Run your idea across the full history and let the trade count tell you how much to trust it.

References

Sample size determination — How wide an estimate stays when the sample is small.
Overfitting