In his recent post, What’s luck got to do with it?, Adam Grimes demonstrated the importance of considering randomness in performance evaluation.  The results, for some, may be quite shocking: with 50 hypothetical traders each taking 250 trades from the same, positive-expectancy trading system, you have results ranging from -40% to +100% returns.  For those familiar with random walks, this is less surprising.  Positive expectancy is great, but variance, it turns out, can be a bitch.

It’s a point we’ve touched upon before, but in a different light.  In particular, we’ve discussed how the choice of when to rebalance (which is akin to “sampling” from a trading system) can dramatically affect performance results.

I read another blog post recently from Cesar Alvarez over at Alvarez Quant Trading on a Heikin-Ashi chart-based strategy.  The strategy design was fairly straight forward: using monthly Heikin-Ashi bars, go long after a positive month and exit the market after a negative month.  Ideally the composition of the Heikin-Ashi bar itself would help filter out a significant amount of whipsaw noise.

The initial results in the post showed promise in the design, especially in equity indices.  In the S&P 500 from 1/1/2006 to 6/30/2014, for example, the Heikin-Ashi strategy had only 11 trades over the period returning 10.10% annualized with a max drawdown of 15.17%.  Compared to the buy & hold statistics of 7.59% annualized return with a 55.17% max drawdown, this strategy appears fairly attractive.

But with monthly sampling, the question is: was it just lucky?

A simple test for this is to “quantify” a month as 21-days and run 21 different versions of the strategy, each rebalancing on a different day.  We can then evaluate the statistics across the strategies to get a better idea of potential strategy performance.

And the results aren't quite as pretty.

rebalance luck

Remember, this is the same strategy just rebalanced at different times.  The results are telling.  If you got lucky and chose the right date of the month to rebalance, you had a 10.02% annualized return; unlucky and you sat at 5.76% – over 400bps below.  And that great drawdown reduction?  17.35% for the best versus 26.97% for the worst.

While the summary statistics are telling, the year-to-year statistics really capture the variance.  In 2011 – when the S&P 500 returned close to 2% – the best performing strategy returned 8.62% whereas the worst returned -13.48%.  On numbers alone, one of those managers you're hiring and the other you're firing.  And they are using the same strategy.

(And the number differentials get much worse the further you take the strategy evaluation back in time).

As a portfolio manager, I almost never receive detailed questions about when or how we go about making rebalance decisions.  But when the result of rebalance decisions can result in a 400bp annualized performance differential due to timing luck (i.e. variance), perhaps it is something we should all spend more time discussing.