In keeping up with my research reading, I came across a white paper from J.P. Morgan titled Defining Absolute Return Investing in Fixed Income.

A graph on one of the last pages caught my eye:

JPMorgan - Defining Absolute Return Investing in Fixed Income

The stunning non-normality of the Barclay's Aggregate Bond index struck me.  "It must be that yield is significantly swamping price return," I thought.

So I went to recreate the results.  I took monthly total returns for the Barclay's Aggregate Bond index going back to 2/1976.

And I got this:

JPMBlog - Figure 1

Well ... that looks different.  Still not normal – but nothing like what had been shown in the exhibit.

Maybe J.P. Morgan has some other information out there about?  So I looked up a classic: J.P. Morgan's The non-normality of market returns.  Within, I found this graph.

JPMorgan – Non-normality of Market Returns

Okay – not normal, but nothing like the first exhibit.  What in the world was going on?

Sometimes it takes fresh eyes to see the problem.  My co-PM Justin Sibears spotted it right away: the bins in the original histogram aren't equally spaced.  Some are 0.25% in width, some are 0.5% in width and some are 1% in width.

Taking this "modified histogram" approach, I recreated the following graph:

JPMBlog - Figure 2

Okay ... whacky, and still not quite the same – but the number of observations seems to be wildly different, so it's likely the period we're looking over.

Fortunately, we can just sum up the number of observations and back out their time frame.  The original graph says "As of December 31, 2014" and has a grand total of 176 observations, meaning their study goes from May, 2000 through the end of 2014.

Let's plot those returns:

JPMBlog - Figure 4

Much, much closer.  But are the returns as "non-normal" as they claim?  Back to an equal-width histogram:

JPMBlog - Figure 3
Still "normalish."  Not perfect, but not the absurd results in the original graph.

But what in the world is going on with the location of the fit normal distribution?  The fits I am getting for the weird bins are obviously wrong, as they aren't appropriately taking into account the density.  Normalizing the weird-bin histogram, we get a better fit:

JPMBlog - Figure 4 NORMED

But there is still something amiss: looking at the original graph, it looks like the mean is around 0.25, whereas the mean in the graph I am plotting is around 0.5.

What has happened?  I believe it was a simple math error.  Often, index data like this is given in linear returns and we have to convert to log returns.  This is done by via the formula log(x+1) where x is the linear return.

Except that most of this index data represents a 1% return as the number 1 instead of the number 0.01.  So you really have to take log(x/100 + 1) (and then multiply by 100 if you want to plot 0.0025 as 0.25).

If you took this extra step, your expected value is 0.46 – if you forgot, you get an expected value of 0.27.  Which I think is what happened here.

While I am sure the fit normal distribution error is an honest mistake, the varying bin sizes seems intentional.  At the very least, if you are going to use varying bin sizes, you should normalize to get a more accurate representation of density.

The lesson is clear: if you're going to be relying on other people's research (which, in a field like finance, is somewhat of a must) – make sure you validate their results.

Corey is co-founder and Chief Investment Officer of Newfound Research, a quantitative asset manager offering a suite of separately managed accounts and mutual funds. At Newfound, Corey is responsible for portfolio management, investment research, strategy development, and communication of the firm's views to clients.

Prior to offering asset management services, Newfound licensed research from the quantitative investment models developed by Corey. At peak, this research helped steer the tactical allocation decisions for upwards of $10bn.

Corey is a frequent speaker on industry panels and contributes to ETF.com, ETF Trends, and Forbes.com’s Great Speculations blog. He was named a 2014 ETF All Star by ETF.com.

Corey holds a Master of Science in Computational Finance from Carnegie Mellon University and a Bachelor of Science in Computer Science, cum laude, from Cornell University.

You can connect with Corey on LinkedIn or Twitter.