Overfitting in Backtests: How to Tell a Plateau From a Lucky Peak

Written by chwoo · Crypto trader, 7 years. Running an automated futures bot since spring 2026. · 2026-07-06 · updated 2026-09-07

Here is an uncomfortable fact: if you test enough parameter combinations, you will always find one with a spectacular backtest. Not because a hidden gem exists, but because randomness guarantees it. With hundreds of combinations of periods, thresholds and exits, some configuration will happen to dodge every crash and catch every rally in your historical data, by pure coincidence.

That is overfitting (or curve-fitting): mistaking a pattern in past noise for a property of the market. The trouble is that the lucky configuration looks exactly like a real one on screen. A return of +400% renders as +400% whether it came from an edge or from chance. There is only one way to tell them apart, and it is to look at the neighbours.

Peaks and plateaus

When you find a setting that looks good, nudge it one step in each direction and run it again. If the RSI period was 14, check 12 and 16. If the threshold was 30, check 25 and 35. One of two things happens.

• Nearby settings behave similarly. That is a plateau, and the strategy plausibly captured something real about market behaviour, because real effects are not razor-thin.
• Performance collapses or flips sign one step away. That is a peak, and you found a coincidence. The market does not care that your RSI period was exactly 13.

Rather than assert it, measure it

We ran the sweep with this site's own engine. The subject is the rule every beginner learns first: go long when RSI crosses above 30. BTC on the 1-hour timeframe, 3% take profit and 2% stop loss, 0.06% fee and 0.02% slippage charged per side, 1x leverage, 90% position size, on Binance candles through 31 August 2026. First we varied only the RSI period.

Open the strategy this guide measures in the backtester →

The lesson here is not that period 20 is the best one. It is that this is neither a peak nor a plateau, because the whole landscape is under water. Every value loses at least two thirds of the account. Picking the single least bad cell from that table and announcing that "RSI 20 is optimal" would not be optimisation; it would be choosing which loss to take.

We checked whether moving the threshold changes that, holding the period at 14.

That is thirteen settings tested and not one of them profitable. The common reaction at this point is to search on a finer grid: try periods 13, 15 and 17, thresholds 28 and 32. Do that long enough and eventually a cell will come back positive. That moment is precisely where overfitting happens.

The multiple-comparisons trap

Flip a thousand fair coins ten times each and a few of them will land heads eight times or more. Pointing at one of those coins and saying "this coin favours heads" is wrong. You did not find a special coin; you had a lot of coins.

Parameter search is exactly that. You are not testing one configuration, you are testing hundreds at once and keeping the best. The best configuration's performance is by definition skill plus luck, and the more attempts you make, the larger the luck term grows. This is why recording what you tried matters: a +50% found on the fortieth attempt is not the same number as a +50% found on the first.

Change the structure, not the parameters

What those tables actually teach is not how to pick better numbers. It is to ask why the rule loses. RSI crossing above 30 keeps firing during downtrends, and each time it catches a falling knife. So instead of shaking the parameters, add a condition: only buy while price is above the 200 EMA.

Same indicator, same values, entirely different outcome. And note what did not change: the win rate is 38.0% in both rows. The filter did not make the strategy win more often. It made it decline trades it was going to lose. Trades fell from 695 to 79 and the loss fell with them. No amount of parameter tuning produces a change of that kind.

This is not where to stop. The filtered version could be a peak too, so it deserves the same neighbour check.

Now the landscape is gentle. Everything sits between -5% and -26%, and nothing collapses or flips sign one step over. It is still a loss everywhere, but a consistent loss is itself information: it says this rule is not unlucky, it is structurally unable to clear its costs.

Watch the sample disappear

The best-looking cell in that last table is -5.4% at period 18. That cell rests on 19 trades. A return computed from nineteen trades is a story, not a statistic; a handful of different outcomes either way would flip its sign.

Every condition you add removes entry opportunities. 695 trades became 79, and lengthening the period took it to 19. So a filter is not free. It buys a smaller loss by spending sample, and once the sample is thin the remaining number drifts back toward chance. That is why this tool warns you once a strategy carries more than about four conditions.

One golden period is not enough

A strategy tuned on 2021 alone learns one lesson: buy the dip. That lesson dies in the next bear market. Robustness means surviving across regimes: up years, down years and sideways chop. It is why this site breaks results into 7-year, 5-year, 3-year, 2-year, 1-year and 6-month windows instead of showing only a total.

The filtered version above was profitable in just 1 of its 8 covered windows. A total loss shrinking to -17% does not mean an edge appeared.

Fewer knobs, more trust

A strategy with two parameters is harder to overfit than one with six, because the search space is smaller. Given equal performance, prefer the one with fewer knobs. And insist on an explanation for why the edge should exist: who is on the other side of the trade, and why they keep giving money away. Without that, all you have is a curve.

A practical checklist

• Check parameter neighbours: plateaus, not peaks.
• Check every period window, not just the total. Consistency beats magnitude.
• Check the trade count. Under about 30 trades, win rate and payoff ratio are not usable even as estimates.
• Check with fees and slippage switched on. A strategy that only works at zero cost is not a strategy.
• Suspect the structure before you shake the parameters. One filter moved more than twenty values did.
• Keep a written log of what you tested. Your future self will not remember the 40 failures.