Backtests have become the weapon of choice for rationalizing various forms of tactical asset allocation, which has become increasingly popular as a risk-management tool since the 2008 crash. The hazards of backtesting—studying how a strategy performed in the past–are well known, which leads some folks to shun the concept entirely. But that’s going too far.

In some respects, every investment plan owes a debt to some type of backtesting—even for a buy-and-hold strategy, which assumes that the future will deliver gains on par with what was earned in the past. The proper lesson is that designing robust backtests, which demands close attention to detail. Easier said than done, of course, in part because the pitfalls can be subtle. Here are three that routinely trip up the novice and perhaps even some experienced investors:

1) the use of total-return prices for technical signals
2) failing to correct for look-ahead bias by not using lagged signals
3) overlooking the importance of neutral signals for computing backtest results

The good news is that these traps are easily avoided. But there’s a catch: you have to be aware of the hazards. With that in mind, let’s briefly review these backtesting snares with some simple examples.

Total return data. Imagine that you’ve created what you think of as a winning investment strategy that’s based on two signals: a) the ratio for a set of short and long moving averages; b) the trailing return for a rolling x-day window. The results look encouraging, but the upbeat outcome may be an illusion if the calculations use total return prices.

Why? Consider a mutual fund that’s unchanged on the day but dispenses a hefty distribution at the close of trading. Imagine that this fund is priced at $10 a share and it spits out a 50-cent-per-share payout. Although the underlying portfolio value was unchanged on the day the mutual fund’s price falls by 50 cents to $9.50 to compensate for the distribution. The net result for shareholders: their holdings in the fund remain unchanged on the day. The 50-cent-per-share drop is offset by a 50-cent distribution. In short, a net wash.

Print Friendly, PDF & Email