Common Backtesting Mistakes That Make Results Look Better Than Reality

The most common backtesting mistakes all do the same thing: they let the test know something the investor could not have known. Future information leaks into past decisions, dead companies vanish from the universe, costs go unmodelled, and the same history gets mined until some variation looks good by luck.

None of these produce an error message. That is precisely the problem. A broken backtest does not crash. It returns a slightly better number, and the better it looks the less anyone questions it.

Here are the recurring errors, roughly in order of how much damage they do.

Mistake 1: Using data that did not exist yet

This is the single biggest source of inflated results, and it hides in ordinary-looking places.

A quarterly result is not public on the last day of the quarter. It is filed weeks later. If a strategy screens on the March quarter’s earnings using a decision date in early April, it is acting on numbers no one had. Every fundamental input needs a knowledge date separate from its period date.

Restated history is the subtler version. Databases routinely overwrite an originally reported figure with the recast version that appears after a demerger, a disposal, or an accounting standard change. Those recast numbers were unknowable at the time, and a test that uses them is quietly reading the future. This is why point-in-time data matters and why restatements break models.

Prices have their own version. Building a signal from a close and then executing at that same close assumes an order placed after the market shut. Building a ranking from a full-period average and applying it to the start of that period does the same thing more invisibly. The general phenomenon has a name, lookahead bias, and it is worth learning to recognise its smell: a result that is inexplicably smooth.

Mistake 2: Building the universe from today’s survivors

Take the current constituents of an index, pull ten years of history for each, and run a strategy. Every company in that list is one that existed, stayed listed, and remained large enough to qualify all the way through to today.

The companies that were delisted, absorbed, or dropped out are missing. Their bad outcomes are excluded from the sample, so the average outcome improves for reasons unrelated to the strategy. Survivorship bias in backtests can lift a result on its own, with no other mistake anywhere in the code.

The correct approach is a historical membership record: what was eligible on each past rebalance date, including names that no longer trade, with their delisting outcome handled explicitly rather than by silent omission.

Mistake 3: Ignoring costs, slippage, and liquidity

A backtest with zero transaction costs is a description of a market that does not exist. Indian equity trades carry brokerage, securities transaction tax, exchange charges, stamp duty, regulatory fees, and GST on top. Then there is the implicit cost of not getting the price you modelled.

The damage is proportional to turnover, so this mistake is fatal for exactly the strategies that look most exciting. A signal that rebalances weekly pays the toll many times more often than one that rebalances annually, and a gross-of-cost comparison between the two is meaningless.

Liquidity is the related blind spot. If a backtest allocates a position that would represent a large share of a stock’s typical daily traded value, the fill assumed in the test could not have happened. A strategy can be entirely real in small, thinly traded names and entirely unusable with real money.

Mistake 4: Testing until something works

Run a hundred variations of a rule on one history and some of them will look excellent. That is arithmetic, not insight. The more parameters you tune, the more thresholds you nudge, and the more times you look at the result and adjust, the more of the final performance is fitted noise.

This is overfitting, and it is insidious because each individual adjustment feels reasonable. You are not cheating. You are just choosing the version that worked, over and over, on the same sample.

Two habits reduce it. First, decide the rules before seeing the result, and count how many variations you have tried. Second, hold data back and evaluate on it once, which is the point of in-sample versus out-of-sample testing and, more strictly, of walk-forward analysis.

Mistake 5: Choosing the benchmark after the fact

A strategy compared against a benchmark selected once the result is known will always look good, because you can shop for the comparison. Related traps include comparing a dividend-reinvesting strategy against a price index that excludes dividends, or comparing a mid-cap strategy against a large-cap index and calling the size effect skill.

Pick the benchmark before you run the test, pick it to match the risk and universe of the strategy, and keep it fixed.

Mistake 6: Reporting only the flattering statistics

A single headline return hides almost everything that matters. So does a single risk-adjusted ratio quoted without the return series behind it.

The statistics to insist on, together, include the worst peak-to-trough fall, the time spent recovering from it, the number of positions and trades, the turnover, the concentration of the result across periods and holdings, and the sensitivity to the cost assumption. Each of these answers a different question, which is what the portfolio metrics hub is organised around.

The specific pattern to look for is concentration of contribution. If the entire outcome comes from a handful of positions in one window, the strategy is a description of that window.

Mistake 7: Testing a period with only one market environment

A test that spans a single long rising market has learned how the rules behave in a rising market. That is real information, but it is not the information most people think they have bought.

You want the sample to contain at least one meaningful fall and one recovery, and ideally more than one type of fall, because a liquidity-driven decline and a slow grinding derating stress a strategy differently. Even then, the sample is finite and the future has no obligation to resemble it.

Mistake 8: Confusing a backtest with evidence

The final mistake is a category error rather than a technical one. A backtest is a simulation of one sample of history under assumptions you selected. It is a filter, not a proof.

Its honest uses are narrow and valuable: eliminating ideas that never worked, revealing the risk and turnover a rule implies, and forcing you to state the strategy precisely enough to test it at all. Its dishonest use is as evidence that something will work.

What careful backtesting still cannot fix

Even a technically flawless backtest carries limits you cannot engineer away.

It cannot tell you whether a pattern has an economic reason to exist. Two rules with identical historical results, one grounded in a plausible mechanism and one found by search, are not equally trustworthy, and the backtest cannot distinguish them.

It cannot model behaviour. Most strategies are abandoned by their owners during a drawdown rather than defeated by the market, and no simulation captures the experience of being underwater for a long stretch.

It cannot represent capacity or the market’s response to the strategy itself. If enough capital follows a rule, the conditions that produced the historical result change.

And it cannot account for the choices you made without noticing: the start date, the universe filter, the handling of a data gap. These decisions are made once, quietly, and they can carry more of the result than the strategy does. That is the honest reason backtest results do not repeat as often as their charts suggest.

Related reading

• Portfolio Metrics Explained: the full set of risk, return, and trade statistics a backtest should report.
• How to Backtest a Stock Strategy in India: the end-to-end process done carefully.
• What Is Overfitting in Backtesting: why more tuning makes results worse, not better.
• Survivorship Bias in Backtests: the companies missing from your universe.
• What Is Lookahead Bias: how future information sneaks into past decisions.

This article is educational. Altys Labs is not a registered research analyst or investment adviser, and nothing here is investment advice or a recommendation to buy, sell, or hold any security.