Overfitting: The "Perfect Settings" You Found Will Fail Live

2026-07-06

Here is an uncomfortable fact: if you test enough parameter combinations, you will always find one with a spectacular backtest. Not because a hidden gem exists, but because randomness guarantees it. With hundreds of combinations of periods, thresholds and exits, some configuration will happen to dodge every crash and catch every rally in your historical data — by pure coincidence.

That is overfitting (or curve-fitting): mistaking a pattern in past noise for a property of the market. It is the second great destroyer of trading accounts, right after look-ahead bias.

Peaks vs. plateaus

The most practical overfitting test costs one minute: look at the neighbors. Say RSI period 13 with a 3.2% take-profit produces a great result. What happens at period 12 and 14? At 3.0% and 3.5%?

If nearby settings are also profitable — a plateau — the strategy plausibly captures something real about market behavior, because real effects are not razor-thin.
If performance collapses or flips negative one step away — a peak — you found a coincidence. The market does not care that your RSI period was exactly 13.

A useful habit: never report the best cell of a parameter grid. Report the neighborhood. A mediocre setting surrounded by other mediocre-but-positive settings beats a brilliant setting surrounded by losses.

The multiple-comparisons trap

Testing many strategies is statistically identical to letting many monkeys throw darts: the best monkey looks like a genius. If you evaluated 200 combinations, the expected "best" result is impressive even when every single strategy is worthless. The more you searched, the higher the bar for believing the winner.

This is why "I tried a bunch of stuff and this one worked" is not evidence. Discoveries need an explanation of why the edge should exist — who is on the other side of the trade, and why they keep giving money away.

One golden period is not enough

A strategy tuned on a single bull year will learn one lesson: "buy every dip." That lesson dies in the next bear market. Robustness means surviving across regimes — up years, down years, and sideways chop.

This site's results table shows returns over 7y / 5y / 3y / 2y / 1y / 6m / 3m / 1m windows for exactly this reason. A strategy that is only positive in one window is a story about that window, not about the strategy. Distrust anything that cannot stay positive across most long horizons.

Fewer knobs, more trust

Every adjustable parameter is another dimension in which noise can be fit. A strategy with two parameters that performs decently is worth more than a strategy with seven parameters that performs beautifully. When adding a filter, demand that it earns its place: it should improve results for a reason you can say out loud, keep most of your trade sample, and work at more than one threshold.

A practical checklist

Check parameter neighbors — plateaus, not peaks.
Check every period window, not just the total. Consistency beats magnitude.
Count how many combinations you tried before this one "worked".
Prefer fewer parameters and filters you can justify in one sentence.
Keep a written log of what you tested — your future self will not remember the 40 failures.

The overfitting warning on this site's results page is not boilerplate. Tweaking settings until the table turns green is the most natural thing in the world — and it is precisely how curve-fit strategies are born. When you find something good, your first move should be trying to break it, not to lever it.