A PDF version of this post is available for download here.
Summary
- In an ideal world, all investors would outperform their benchmarks. In reality, outperformance is a zero-sum game: for one investor to outperform, another must underperform.
- If achieving outperformance with a certain strategy is perceived as being “easy,” enough investors will pursue that strategy such that its edge is driven towards zero.
- Rather, for a strategy to outperform in the long run, it has to be hard enough to stick with in the short run that it causes investors to “fold,” passing the alpha to those with the fortitude to “hold.”
- In other words, for a strategy to outperform in the long run, it must underperform in the short run. We call this The Frustrating Law of Active Management.
A few weeks ago, AQR published a piece titled Craftsmanship Alpha: An Application to Style Investing[1], to which Cliff Asness wrote a further perspective piece titled Little Things Mean a Lot[2].
We’ll admit that we are partial to the title “craftsmanship alpha” because portfolio craftsmanship is a concept we spend a lot of time thinking about. In fact, we have a whole section titled Portfolio Craftsmanship on the Investment Philosophy section of our main website.[3] We further agree with Cliff: little things do mean a lot. We even wrote a commentary about it in May titled Big Little Details[4].
But there was one quote from Cliff, in particular, that inspires this week’s commentary:
Let’s just make up an example. Imagine there are ten independent (uncorrelated) sources of “craftsmanship alpha” and that each adds 2 basis points of expected return at the cost of 20 basis points of tracking error from each (against some idea of a super simple “non-crafted” alternative.) Each is thus a 0.10 Sharpe ratio viewed alone. Together they are expected to add 20 basis points to the overall factor implementation inducing 63 basis points of tracking error (20 basis points times the square-root of ten). That’s a Sharpe ratio of 0.32 from the collective craftsmanship (in addition to the basic factor returns).
[…]
But, as many have noted in other contexts, a Sharpe ratio like 0.32 can be hard to live with. Its chance of subtracting from your performance in a given year is about 37%. Its chance of subtracting over five years is about 24%. And, wait for it… over twenty years the chance it subtracts is still about 8%. That’s right. There’s a non-trivial chance your craftsmanship is every bit as good as you think, and it subtracts over two full decades, perhaps the lion’s share of your career. Such is the unforgiving, uncaring math.
Whether it is structural alpha, style premia, or craftsmanship alpha: we believe that the very uncertainty and risk that manifests as (expected) tracking error is a necessary component for the alpha to exist in the first place.
The “unforgiving, uncaring math” that is a result – the fact that you can do everything right and still get everything wrong – is a concept that in the past we have titled The Frustrating Law[5] of Active Management.
Defining The Frustrating Law of Active Management
We define The Frustrating Law of Active Management as:
For any disciplined[6] investment approach to outperform over the long run, it must experience periods of underperformance in the short run.
As if that were not frustrating enough a concept – that even if we do everything right, we still have to underperform from time-to-time – we add this corollary:
For any disciplined investment approach to underperform over the long run, it must experience periods of outperformance in the short run.
In other words, even if a competing manager does everything wrong, they should still be rewarded with outperformance at some point. Talk about adding insult to injury.
For the sake of brevity, we will only explore the first half of the law in this commentary. Note, however, that the second law is simply the inverse case of the first. After all, if we found an investment strategy that consistently underperformed, we could merely inverse the signals and have a strategy that consistently outperforms. If the latter is impossible, so must be the former.
For it to work, it has to be hard
Let’s say we approach you with a new investment strategy. We’ve discovered the holy grail: a strategy that always outperforms. It returns an extra 2% over the market, consistently, every year, after fees.
Ignoring reasonable skepticism for a moment, would you invest? Of course you would. This is free money we’re talking about here!
In fact, everyone we pitch to would invest. Who wouldn’t want to be invested in such a strategy? And here, we hit a roadblock.
Everyone can’t invest. Relative performance is, after all, zero sum: for some to outperform, others must underperform. Our extra return has to come from somewhere.
If we do continue to accept money into our strategy, we will begin to approach and eventually exceed capacity. As we put money to work, we will create impact and inform the market, driving prices away from us. As we try to buy, prices will be driven up and as we try to sell, prices will be driven down. By chasing price, our outperformance will deteriorate.
And it needn’t even be us trading the strategy. Once people learn about what we are doing – and how easy it is to make money – others will begin to employ the same approach. Increasing capital flow will continue to erode the efficacy of the edge as more and more money chases the same, limited opportunities. The growth is likely to be exponential, quickly grinding our money machine quickly to a halt.
So, the only hope of keeping a consistent edge is in a mixture of: (1) keeping the methodology secret, (2) keeping our deployed capital well below capacity, and (3) having a structural moat (e.g. first-mover advantage, relationship-driven flow, regulatory edge, non-profit-seeking counter-party, etc).
While we believe that all asset managers have the duty to ensure #2 remains true (we highly recommend reading Alpha or Assets by Patrick O’Shaughnessy[7]), #1 pretty much precludes any manager actually trying to raise assets (with, perhaps, a few limited exceptions in the hedge fund world that can raise assets on brand alone).
The takeaway here is that if an edge is perceived as being easy to implement (i.e. not case #3 above) and easy to achieve, enough people will do it to the point that the edge is driven to zero.
Therefore, if an edge is known by many (e.g. most style premia like value, momentum, carry, defensive, trend, etc), then for it to persist over the long run, the outperformance must be difficult to capture. Remember: for outperformance to exist, weak hands must at some point “fold” (be it for behavioral or risk-based reasons), passing the alpha to strong hands that can “hold.”
This is not just a case of perception, either. Financial theory tells us that a strategy cannot always outperform its benchmark with certainty. After all, if it did, we would have an arbitrage: we could go long the strategy, short the benchmark, and lock in certain profit. As markets loathe (or, perhaps, love) arbitrage, such an opportunity should be rapidly chased away. Thus, for a disciplined strategy to generate alpha over the long run, it must go through periods of underperformance in the short-run.
Can We Diversify Away Difficulty?
Math tells us that we should be able to stack the benefits of multiple, independent alpha sources on top of each other and simultaneously benefit from potentially reduced tracking error due to diversification.
Indeed, mathematically, this is true. It is why diversification is known as the only free lunch in finance.
This certainly holds for beta, which derives its value from economic activity. In theory, everyone can hold the Sharpe ratio optimal portfolio and introduce cash or leverage to hit their appropriate risk target.
Alpha, on the other hand, is explicitly captured from the hands of other investors. Contrary to the Sharpe optimal portfolio, everyone cannot hold the Information ratio optimal portfolio at the same time[8]. Someone needs to be on the other side of the trade.
Consider three strategies that all outperform over the long run: strategy A, strategy B, and strategy C. Does our logic change if we learn that strategy C is simply 50% strategy A plus 50% strategy B? Of course not! For C to continue to outperform over the long run, it must remain sufficiently difficult to stick with in the short-run that it causes weak hands to fold.
Conclusion
For a strategy to outperform in the long run, it has to be perceived as hard: hard to implement or hard to hold. For public, liquid investment styles that most investors have access to, it is usually a case of the latter.
This law is underpinned by two facts. First, relative performance is zero-sum, requiring some investors to underperform for others to outperform. Second, consistent outperformance violates basic arbitrage theories.
While coined somewhat tongue-in-cheek, we think this law provides an important reminder to investors about reasonable expectations. As it turns out, the proof is not always in the eating of the pudding. In fact, track records can be entirely misleading as validators of an investment process. As Cliff pointed out, even if our alpha source has a Sharpe ratio of 0.32, there is an 8% chance that it subtracts from performance over the next 20-years.
Conversely, even negative alpha sources can show beneficial performance by chance. An alpha source with a Sharpe ratio of -0.32 has an 8% chance that it adds to performance over the next 20-years.
And that’s why we call it The Frustrating Law of Active Management. For investors and asset managers alike, there is little more frustrating than knowing that to continue working over the long run, good strategies have to do poorly, and poor strategies have to do well over shorter timeframes.
[1] https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3034472
[2] https://www.aqr.com/cliffs-perspective/little-things-mean-a-lot
[3] https://www.thinknewfound.com/investment-philosophy
[4] https://blog.thinknewfound.com/2017/05/big-little-details/
[5] To be clear that we don’t mean a “law” in the sense of an inviolable, self-evident axiom. In truth, our “law” is much closer to a “theory.”
[6] The disciplined component here is very important. By this, we mean a strategy that applies a consistent set of rules. We do not mean, here, a bifurcation of systematic versus discretionary. Over the years, we’ve met a large number of discretionary managers who apply a highly disciplined approach. Rather, we mean those aspects of an investment strategy that can be codified and turned into a set of systematically applied rules.
Thus, even a discretionary manager can be thought of as a systematic manager plus a number of idiosyncratic deviations from those rules. The deviations must be idiosyncratic, by nature. If there was a consistent reason for making the deviations, after all, the reason could be codified itself. Thus, true discretion only applies to unique, special, and non-repeatable situations.
Note that the discipline does not preclude randomness. You could, for example, flip a coin and use the result to make an investment decision every month. So long as the same set of rules is consistently applied, we believe The Frustrating Law of Active Management applies.
[7] http://investorfieldguide.com/alpha-or-assets/
[8] Well, technically they can if everyone is a passive investor. In this case, however, the information ratio would be undefined, with zero excess expected return and zero tracking error.
How Much Accuracy Is Enough?
By Nathan Faber
On March 4, 2019
In Craftsmanship, Portfolio Construction, Trend, Weekly Commentary
Available as a PDF download here.
Summary
The distinction between luck and skill in investing can be extremely difficult to measure. Seemingly good or bad strategies can be attributable to either luck or skill, and the truth has important implications for the future prospects of the strategy.
Source: Grinold and Kahn, Active Portfolio Management. (New York: McGraw-Hill, 1999).
Time is one of the surest ways to weed out lucky strategies, but the amount of time needed to make this decision with a high degree of confidence can be longer than we are willing to wait. Or, sometimes, even longer than the data we have.
For example, in order to be 95% confident that a strategy with a 7% historical return and a volatility of 15% has a true expected return that is greater than a 2% risk-free rate, we would need 27 years of data. While this is possible for equity and bond strategies, we would have a long time to wait in order to be confident in a Bitcoin strategy with these specifications.
Even after passing that test, however, that same strategy could easily return less than the risk-free rate over the next 5 years (the probability is 25%).
Regardless of the skill, would you continue to hold a strategy that underperformed for that long?
In this commentary, we will use a sample U.S. sector strategy that isolates luck and skill to explore the impacts of varying accuracy and how even increased accuracy may only be an idealized goal.
The (In)Accurate Investor
To investigate the historical impact of luck and skill in the arena of U.S. equity investing, we will consider a strategy that invests in the 30 industries from the Kenneth French Data Library.
Each month, the strategy independently evaluates each sector and either holds it or invests the capital at the risk-free rate. The term “evaluates” is used loosely here; the evaluation can be as simple as flipping a (potentially biased) coin.
The allocation allotted to each sector is 1/30th of the portfolio (3.33%). We are purposely not reallocating capital among the sectors chosen so that the sector calls based on the accuracy straightforwardly determine the performance.
To get an idea for the bounds of how well – or poorly – this strategy would have performed over time, we can consider three investors:
The Perfect and Anti-Perfect investors set the bounds for what performance is possible within this framework, and the Plain Investor denotes the performance of not making any decisions.
The growth of each boundary strategy over the entire time period is a little outrageous.
Source: Kenneth French Data Library. Calculations by Newfound Research. Past performance is not a guarantee of future results. All returns are hypothetical and backtested. Returns are gross of all fees. This does not reflect any investment strategy offered or managed by Newfound Research and was constructed exclusively for the purposes of this commentary. It is not possible to invest in an index.
A more informative illustration is the rolling annualized 5-year return for each strategy.
While the spread between the Perfect and Anti-Perfect investors ebbs and flows, its median value Is 59,000 basis points (“bps”). Between the Perfect and Plain investors, there is still 29,000 bps of annualized outperformance to be had. A natural wish is to make calls that harvest some of this spread.
Accounting for Accuracy
Now we will look at a set of investors who are able to evaluate each sector with some known degree of accuracy.
For each accuracy level between 0% and 100% (i.e. our Anti-Perfect and Perfect investors, respectively), we simulate 1,000 trials and look at how the historical results have played out.
A natural starting point is the investor who merely flips a fair coin for each sector. Their accuracy is 50%.
The chart below shows the rolling 5-year performance range of the simulated trials for the 50% Accurate Investor.
Source: Kenneth French Data Library. Calculations by Newfound Research. Past performance is not a guarantee of future results. All returns are hypothetical and backtested. Returns are gross of all fees. This does not reflect any investment strategy offered or managed by Newfound Research and was constructed exclusively for the purposes of this commentary. It is not possible to invest in an index.
In 59% of the rolling periods, the buy-and-hold Plain Investor beat even the best 50% Accurate Investor. The Plain Investor was only worse than the worst performing coin flip strategy in 6% of rolling periods.
Beating buy-and-hold is hard to do reliably if you rely only on luck.
In this case, having a neutral hit rate with the negative skew of the sector equity returns leads to negative information coefficients. Taking more bets over time and across sectors did not help offset this distributional disadvantage.
So, let’s improve the accuracy slightly to see if the rolling results improve. Even with negative skew (-0.42 median value for the 30 sectors), an improvement in the accuracy to 60% is enough to bring the theoretical information coefficient back into the positive realm.
Source: Kenneth French Data Library. Calculations by Newfound Research. Past performance is not a guarantee of future results. All returns are hypothetical and backtested. Returns are gross of all fees. This does not reflect any investment strategy offered or managed by Newfound Research and was constructed exclusively for the purposes of this commentary. It is not possible to invest in an index.
The worst of these more skilled investors is now beating the Plain Investor in 41% of the rolling periods, and the best is losing to the buy-and-hold investor in 13% of the periods.
Going the other way, to a 40% accurate investor, we find that the best one was beaten by the Plain investor 93% of the time, and the worst one never beats the buy-and-hold investor.
Source: Kenneth French Data Library. Calculations by Newfound Research. Past performance is not a guarantee of future results. All returns are hypothetical and backtested. Returns are gross of all fees. This does not reflect any investment strategy offered or managed by Newfound Research and was constructed exclusively for the purposes of this commentary. It is not possible to invest in an index.
If we only require a modest increase in our accuracy to beat buy-and-hold strategies over shorter time horizons, why isn’t diligently focusing on increasing our accuracy an easy approach to success?
In order to increase our accuracy, we must first find a reliable way to do so: a task easier said than done due to the inherent nature of probability. Something having a 60% probability of being right does not preclude it from being wrong for a long time. The Law of Large Numbers can require larger numbers than our portfolios can stand.
Thus, even if we have found a way that will reliably lead to a 60% accuracy, we may not be able to establish confidence in that accuracy rate. This uncertainty in the accuracy can be unnerving. And it can cut both ways.
A strategy with a hit rate of less than 50% can masquerade as a more accurate strategy simply for lack of sufficient data to sniff out the true probability.
Source: Kenneth French Data Library. Calculations by Newfound Research. Past performance is not a guarantee of future results. All returns are hypothetical and backtested. Returns are gross of all fees. This does not reflect any investment strategy offered or managed by Newfound Research and was constructed exclusively for the purposes of this commentary. It is not possible to invest in an index.
You may think you have an edge when you do not. And if you do not have an edge, repeatedly applying it will lead to worse and worse outcomes.2
Accuracy Schmaccuracy
Our preference is to rely on systematic bets, which generally fall under the umbrella of factor investing. Even slight improvements to the accuracy can lead to better results when applied over a sufficient breadth of investments. Some of these factors also alter the distribution of returns (i.e. the skew) so that accuracy improvements have a larger impact.
Consider two popular measures of trend, used as the signals to determine the allocations in our 30 sector US equity strategy from the previous sections:
These strategies have volatilities in line with the Perfect and Anti-Perfect Investors and returns similar to the Plain Investor.
Using our measure of accuracy as correctly calling the direction of the sector returns over the subsequent month, it might come as a surprise that the accuracies for the 12-1 Momentum and 10-month SMA signals are only 42% and 41%, respectively.
Even with this low accuracy, the following chart shows that over the entire time period, the returns of these strategies more closely resemble those of the 55% Accurate Investor and have even looked like those of the 70% Accurate Investor over some time periods. What gives?
This is an example of how addressing the negative skew in the underlying asset returns can offset a sacrifice in accuracy. These trend following strategies may have overall accuracy of less than 50%, but they have been historically right when it counts.
Consistently removing large negative returns – at the expense of giving up some large positive returns – is enough to generate a return profile that looks much like a strategy that picks sectors with above average accuracy.
Whether investors can stick with a strategy that exhibits below 50% accuracy, however, is another question entirely.
Conclusion
While most investors expect the proof to be in the eating of the pudding, in this commentary we demonstrate how luck can have a meaningful impact in the determination of whether skill exists. While skill should eventually differentiate itself from luck, the horizon over which it will do so may be far, far longer than most investors suspect.
To explore this idea, we construct portfolios comprised of all thirty industry groups. We then simulate the results of investors with known accuracy rates, comparing their outcomes to 100% Accuracy, 100% Inaccurate, and Buy-and-Hold benchmarks.
Perhaps somewhat counter-intuitively, we find that an investor exhibiting 50% accuracy would have fairly reliably underperformed a Buy-and-Hold Investor. This seems somewhat counter-intuitive until we acknowledge that equity returns have historically exhibit negative skew, with the left tail of their return distribution (“losses”) being longer and fatter than the right (“gains”). Combining a neutral hit rate with negative skew creates negative information coefficients.
To offset this negative skew, we require increased accuracy. Unfortunately, even in the case where an investor exhibits 60% accuracy, there are a significant number of 5-year periods where it might masquerade as a strategy with a much higher or lower hit-rate, inviting false conclusions.
This is all made somewhat more confusing when we consider that a strategy can have an accuracy rate below 50% and still be successful. Trend following strategies are a perfect example of this phenomenon. The positive skew that has been historically exhibited by these strategies means that frequently inaccurate trades of small magnitude are offset by infrequent, by very large accurate trades.
Yet if we measure success by short-term accuracy rates, we will almost certainly dismiss this type of strategy as one with no skill.
When taken together, this evidence suggests that not only might it be difficult for investors to meaningfully determine the difference between skill and luck over seemingly meaningful time horizons (e.g. 5 years), but also that short-term perceptions of accuracy can be woefully misleading for long-term success. Highly accurate strategies can still lead to catastrophe if there is significant negative skew lurking in the shadows (e.g. an ETF like XIV), while inaccurate strategies can be successful with enough positive skew (e.g. trend following).