ETFs provide a vast array of opportunities for investors to implement their investment goals efficiently.  Low fees, inherent diversification, ease of trading, and tax efficiency all make this possible.  However, it is interesting to take a step back and examine how effective ETFs are at achieving their own goals, namely tracking their index.

Tracking error is commonly discussed for mutual funds because the manager’s actions are always present in our minds.  Although ETFs are traded like stocks, they still hold a basket of securities.  Some ETFs have more leeway in how they track their index, meaning they do not always hold exactly what the index holds.  Even if they do mimic the index, certain rules about how ETFs are managed can lead to tracking error.

The tracking error, ψ, is computed as the annualized standard deviation of the daily return difference of the ETF and its benchmark:

ψ = stdev(r_ETF-r_index) 

Morningstar also uses a metric called tracking difference, which is the difference between the annual returns of the ETF and the benchmark.  This metric captures the sign and magnitude of the deviation of the ETF from the index over a longer period of time, while the tracking error represents the day-to-day volatility of the absolute return difference.

With an ETF, we have three prices to consider:

1.  Index price – the idealized, no-fee, no-cost, price of the index
2.  ETF NAV – the price of the ETF provider’s attempt at replicating the index
3.  ETF price – the price that we pay for the ETF provider’s attempt

The price of an ETF can vary from its NAV, sometimes significantly.  However, for large ETFs that trade liquid instruments, the price should remain close to the NAV to wipe out arbitrage opportunities.  The following graph shows the S&P 500 index, along with the ETFs SPY and IVV, from State Street SPDR and Blackrock’s iShares, respectively, and the NAV of SPY.

Pic1Having trouble seeing the different lines?  It’s tough when the ETFs accomplish their goal and closely track the index.

Pic2The tracking error for these ETFs is very low (I’ll put it in context soon but for now, trust me).   The initial larger tracking errors stem from the financial crisis when SPY and IVV underwent days with volatile returns relative to the S&P 500.  Differences in tracking error between the two ETFs can often be explained by how the ETF is managed:

1.  The small blips that appear every few months in the SPY NAV indicate times when the ETF incorporated any changes to the S&P 500 index constituents.
2.  Differences in SPY and its NAV arise because SPY holds dividends as cash (cash drag) until the quarterly distribution date.
3.  SPY purely holds the S&P 500 securities with no room for optimization.  IVV can choose to hold a subset of the S&P 500 securities and can lend out shares (e.g. to short sellers) in exchange for a lending fee.

Despite these differences, we can see that buying exposure to the S&P 500 through ETFs gets you very close to buying the index.

Now, let’s look at two indices that involve a bit more work to track.  The S&P Developed Ex-US BMI Index and the MSCI All Country World ex-US Index both track the performance of a global set of securities.  The MSCI index includes companies in both emerging and foreign developed markets whereas the S&P index is purely companies in foreign developed countries.  The State Street ETF GWL tracks the S&P index, and the ETFS CWI (State Street) and ACWX (iShares) track the MSCI index.

The following two charts show the growth GWL, its NAV, and the underlying index and the annual tracking difference.

Pic3

Pic4

Aside from underperformance coming out of the financial crisis, GWL’s return has hovered around its NAV.  Its NAV has generally returned more than the benchmark index, indicating that some form of portfolio optimization technique is being used to track the index.

The tracking difference for the MSCI index ETFs is shown below:

Pic5

Both ETFs appear to return about as much as their NAV, but CWI has performed slightly better than ACWX (mean tracking difference of -0.5% compared to -1.5%).   This fact is not visible from looking at the tracking error alone.

The tracking error of all 3 of the global ETFs and their NAVs is shown below.

Pic6

Compared to SPY, with an NAV tracking error of~1%, these ETFs have a higher tracking error, generally between 1.5-3%.  The similarity of the tracking error for ACWX and CWI in light of different tracking differences highlights an important fact.

Tracking error alone cannot tell us how an investment will perform relative to a benchmark.  It can only tell us that the investment will be more volatile.  It is also possible to have low tracking error and not match the index at all (consider an extreme case of a fund that consistently beats the benchmark).

Tracking error alone cannot tell us how an investment will perform relative to a benchmark.

The bottom line is that there are multiple ways to evaluate how effectively an ETF tracks its underlying index.  Not all tracking error is bad; its effect on portfolio performance hinges on the direction of the deviations.

By peering behind the curtain, we can become more knowledgeable about how our investment vehicles are managed and gain more confidence that they will function well in both tactical and strategic investment strategies.  It is one thing to suffer a loss due to a given macroeconomic environment - the nature of investing makes this unavoidable - but underperformance due to ambiguous ETF/mutual fund management can limit the reliability of a strategy to perform as intended, during good times and bad.

Nathan is a Vice President at Newfound Research, a quantitative asset manager offering a suite of separately managed accounts and mutual funds. At Newfound, Nathan is responsible for investment research, strategy development, and supporting the portfolio management team.

Prior to joining Newfound, he was a chemical engineer at URS, a global engineering firm in the oil, natural gas, and biofuels industry where he was responsible for process simulation development, project economic analysis, and the creation of in-house software.

Nathan holds a Master of Science in Computational Finance from Carnegie Mellon University and graduated summa cum laude from Case Western Reserve University with a Bachelor of Science in Chemical Engineering and a minor in Mathematics.