How Many Trades Does a Backtest Need? We Resampled 2,126

Written by chwoo · Crypto trader, 7 years. Running an automated futures bot since spring 2026. · 2026-07-20 · updated 2026-09-08

Here is an uncomfortable experiment: flip a fair coin ten times. Reasonably often you will get seven heads. Nobody would conclude the coin is magic. Yet traders routinely see a strategy win 7 of its 10 backtested trades and conclude they found a 70%-win-rate system.

The usual response is to quote the square-root rule and move on. We would rather measure it. So we took a strategy with a large, unambiguous result, Bollinger mean reversion on BTC hourly candles, which makes 2,126 trades, wins 39.2% of them and loses 98.1% of the account, and then asked what a smaller sample of those same trades would have told you.

How much fog sits around a win rate

For each sample size we drew 2,000 random samples from those 2,126 real trades, without replacement, and recorded the win rate of each. The 5th and 95th percentile columns are the band that 90% of samples fell inside. The true value is 39.2% throughout.

Open the strategy this guide measures in the backtester →

At ten trades the band runs from 20.0% to 60.0%. The same strategy, the same market, the same years; only which ten trades you happened to look at. A 60% win rate and a 20% win rate are both ordinary outcomes of a 39.2% system.

The widths do roughly follow the square-root rule. Going from 10 trades to 100 is ten times the sample and cuts the band from 40.0 to 15.0 points, close to the factor of about three the rule predicts; another five times the sample, to 500, brings it to 6.4. Precision is expensive: each halving of the fog costs about four times the trades.

A strategy that loses 98% looks profitable one time in three

Win rate is the polite version of the problem. Here is the same experiment reading the return of each sample instead, compounded at this site's default 90% position size.

Read the last column downward. A strategy that destroys 98.1% of the account shows a profit in 35.9% of ten-trade samples and 19.3% of hundred-trade samples. It takes a thousand trades before that number reaches zero. If you test ten ideas on twenty trades each, you should expect several of them to look like edges. That is not bad luck; that is the arithmetic of the test you chose.

The wobble in that column, 35.9% at ten trades rising to 38.4% at twenty and dropping to 26.3% at thirty, is itself sampling noise, and worth pointing at. Even a measurement made from 2,000 draws is not perfectly smooth. Nothing about small numbers is stable, including statements about small numbers.

What cherry-picking can buy you

The samples above were random. A person choosing a screenshot is not random. So we slid a window of N consecutive trades along the whole history and kept the best and worst window at each length.

The best thirty-trade stretch of a strategy that loses 98.1% returned +43.1%, winning 70% of its trades. The best ten-trade stretch won 80%. Every number in that row is real, measured, reproducible, and completely misleading. This is what a marketing screenshot is made of, and no fabrication is required to produce one.

The bottom row is the useful part. At 500 consecutive trades not one window in 1,627 is profitable; the best available stretch is -22.8%. Somewhere between 200 and 500 trades, this strategy runs out of places to hide. That is a concrete answer to the question in the title, for this strategy: a few hundred trades is where cherry-picking stops working.

Trade count is not the same as robustness

We expected the strategies with few trades to be the ones whose period windows disagree most, and went to check. The data did not support it.

The 40-trade golden cross has the widest gap of all at 151.3 points, which fits the expectation. But the 2,126-trade Bollinger strategy sits at 105.1 points and the 1,743-trade MACD at 116.2, no better. And the smallest gap in the table, 7.1 points, belongs to a strategy with only 80 trades.

The explanation is that these two columns measure different things. A window gap is partly the size of the returns involved: the trend-filtered RSI barely moves in any window, so its windows cannot disagree by much. Sample size bounds the noise in an estimate, which the first two tables demonstrate cleanly. It does not make different eras agree. Enough trades and consistency across regimes are two separate requirements, and a strategy can pass either one while failing the other.

The check that did work

If a large sample does not settle it, what does? Running the same rules somewhere else. Here is the same Bollinger strategy, unchanged, on five coins.

Five markets, more than 14,000 trades, one answer. Compare that with the +43.1% window from earlier and the difference between evidence and an anecdote is visible. An out-of-sample test costs one click and answers a question a bigger in-sample test cannot.

What to do with this

• Under about 30 trades, do not form an opinion. The measured band at ten trades was 40 points wide around the true win rate.
• Treat a few hundred trades as the entry price for a claim. That is where cherry-picking stopped working above, and where the win-rate band narrowed to single digits.
• Expect a third of your small tests to look good by accident. If you tried ten ideas on twenty trades, the best of them tells you nothing.
• Read the period table, not just the total. A strategy whose windows contradict each other has a problem no trade count fixes.
• Re-run on other coins and timeframes. This was the cheapest and most decisive check in the whole sweep.