This post is available as a PDF download here.
Summary
- Using historical data from the options market along with realized subsequent returns, we can translate risk-neutral probabilities into real-world probabilities.
- Market implied probabilities are risk-neutral probabilities derived from the derivatives market. They incorporate both the probability of an event happening and the equilibrium cost associated with it.
- Since investors have the flexibility of designing their own portfolios, real-world probabilities of event occurrences are more relevant to individuals than are risk-neutral probabilities.
- With a better handle on the real-world probabilities, investors can make portfolio decisions that are in line with their own goals and risk tolerances, leaving the aggregate decision making to the policy makers.
Market-implied probabilities are just as the name sounds: weights that the market is assigning an event based upon current prices of financial instruments. By deriving these probabilities, we can gain an understanding of the market’s aggregate forecast for certain events. Fortunately, the Federal Reserve Bank of Minneapolis provides a very nice tool for visualizing market-implied probabilities without us having to derive them.[1]
For example, say that I am concerned about inflation over the next 5 years. I can see how the probability of a large increase has been falling over time and how the probability of a large decrease has fallen recently, with both currently hovering around 15%.
Historical Market Implied Probabilities of Large Moves in Inflation
Source: Minneapolis Federal Reserve
I can also look at the underlying probability distributions for these predictions, which are derived from the derivatives market, and compare the changes over time.
Market Implied Probability Distributions for Moves in Inflation
Source: Minneapolis Federal Reserve
From this example, we can judge that not only has the market’s implied inflation forecast increased, but the precision has also increased (i.e. lower standard deviation) and the probabilities have been skewed to the left with fatter tails (i.e. higher kurtosis).
Inflation is only one of many variables analyzed.
Also available is implied probability data for the S&P 500, short and long-term interest rates, major currencies versus the U.S. dollar, commodities (energy, metal, and agricultural), and a selection of the largest U.S. banks.
With all the recent talk about low volatility, the data for the S&P 500 over the next 12 months is likely to be particularly intriguing to investors and advisors alike.
Historical Market Implied Probabilities of Large Moves in the S&P 500
Source: Minneapolis Federal Reserve
The current market implied probabilities for both large increases and decreases (i.e. greater than a 20% move) are the lowest they have been since 2007.
Interpreting Market Implied Probabilities
A qualitative assessment of probability is generally difficult unless the difference is large. We can ask ourselves, for example, how we would react if the probability of a large loss jumped from 10% to 15%. We know that the latter case is riskier, but how does that translate into action?
The first step is understanding what the probability actually means.
Probability forecasts in weather are a good example of this necessity. Precipitation forecasts are a combination of forecaster certainty and coverage of the likely precipitation.[2] For example, if there is a 40% chance of rain, it could mean that the forecaster is 100% certain that it will rain in 40% of the area. Or it could mean that they are 40% certain that it will rain in 100% of the area. Or it could mean that they are 80% certain that it will rain in 50% of the area.
Once you know what the probability even represents, you can have a better grasp on whether you should carry an umbrella.
In the case of market implied probabilities, what we have is the risk-neutral probability. These are the probabilities of an event given that investors are risk neutral; these probabilities factor in the both the likelihood of an event and the cost in the given state of the world. These are not the real-world probabilities of the market moving by a given amount. In fact, they can change over time even if the real-world probabilities do not.
To illustrate these differences between a risk-neutral probability and a real-world probability, consider a simple coin flip game. The coin is flipped one time. If it lands on heads, you make $1, and if it lands on tails, you lose $1.
The coin is fair, so the probability for the coin flip is 50%. How much would you pay to play this game?
If you answer is nothing, then you are risk neutral, and the risk neutral probability is also 50%.
However, risk averse players would say, “you have to pay me to play that game.” In this case, the risk neutral probability of a tails is greater than 50% because of the downside that players ascribe to that event.
Now consider a scenario where a tails still loses $1, but a heads pays out $100. Chances are that even very risk-averse players would pay close to $1 to play this game.
In this case, the risk neutral probability of a heads would be much greater than 50%.
But in all cases, the actual likelihoods of heads and tails never changed; they still had a 50% real-world probability of occurring.
As with the game, investors who operate in the real world are generally risk averse. We pay premiums for insurance-like investments to protect in the states of the world we dread the most. As such, we would expect the risk-neutral probability of a “bad” event (e.g. the market down more than 20%) to be higher than the real-world probability.
Likewise, we would expect the risk-neutral probability of a “good” event (e.g. the market up more than 20%) to be lower than the real-world probability.
How Market Implied Probabilities Are Calculated
Note (or Warning): This section contains some calculus. If that is not of interest, feel free to skip to the next section; you won’t miss anything. For those interested, we will briefly cover how these probabilities are calculated to see what (or who), exactly, in the market implies them.
The options market contains call and put options over a wide array of strike prices and maturities. If we assume that the market is free from arbitrage, we can transform the price of put options into call options through put-call parity.[3]
In theory, if we knew the price of a call option for every strike price, we could calculate the risk-neutral probability distribution, fRN, as the second derivative with respect to the strike price.
where r is the risk-free rate, C is the price of a call option, K is the strike price and T-t is the time to option maturity.
Since options do not exist at every strike price, a curve is fit to the data to make it a continuous function that can be differentiated to yield the probability distribution.
Immediately, we see that the probabilities are set by the options market.
Are Market Implied Probabilities Useful?
Feldman et. al (2015), from the Minneapolis Fed, assert that market-based probabilities are a useful tool for policy makers.[4] Their argument centers around that fact that risk-neutral probabilities encapsulate both the probability of an event occurring – the real-world probability – and the cost/benefit of the event.
Assuming broad access to the options market, households or those acting on behalf of households can express their views on the chances of the event happening and the willingness to exchange cash flows in different states of the world by trading the appropriate options.
In the paper, the authors admit two main pitfalls:
- Participation – An issue can arise here since the people whose welfare the policy-makers are trying to consider may not be participating. Others outside the U.S. may also be influencing the probabilities.
- Illiquidity – Options do not always trade frequently enough in the fringes of the distribution where investors are usually most concerned. Because of this, any extrapolation must be robust.
However, they also refute many common arguments against using risk-neutral probabilities.
- These are not “true” probabilities – The fact that these market implied probabilities are model-independent and derived from household preferences rather than from a statistician’s model, with its own biased assumptions, is beneficial, especially since these market probabilities account for resource availability.
- No Household is “Typical” – In equilibrium, all households should be willing to rearrange their cash flows in different states of the world as long as the market is complete. Therefore, a policy-maker aligns their beliefs with those of the households in aggregate by using the market-based probabilities.
We have covered how policymakers often do not forecast very well themselves[5], which Ellison and Sargent argue may be intentional, stating that the FOMC may purposefully forecast incorrectly in order to form policy that is robust to model misspecification.[6]
Where a problem could arise is when an individual investor (i.e. a household) makes a decision for their own portfolio based on these risk-neutral probabilities.
We agree that having a financial market makes a “typical” investor more relevant than the “average fighter pilot” example in our previous commentary.[7] But what a central governing body uses to make decisions is different from what may be relevant to an individual.
The ability to be flexible is key. In this case, an investor can construct their own portfolio. It would be like a pilot constructing their own plane.
Getting to Real World Probabilities
Using the method outlined in Vincent-Humphreys and Noss (2012), we can transform risk-neutral probabilities into real-world probabilities, assuming that investor risk preferences are stable over time.[8]
Without getting too deep into the mathematical framework, the basic premise is that if we have a probability density function (PDF) for the risk-neutral probability, fRN, with a cumulative density function (CDF), FRN, we can multiply it by a calibration function, C, to obtain the real-world probability density function, fRW.
The beta distribution is a suitable choice for the calibration function.[9] Using a beta distribution balances being parsimonious – it only has two parameters – with flexibility, since it allows for preserving the risk-neutral probability distribution by simply shifting the mean and adjusting the variance, skew, and kurtosis.
The beta distribution parameters are calculated using the realized value that the market implied probability represents (e.g. change in the S&P 500, interest rates commodity prices, etc.) over the subsequent time period.
Deriving the Real-World Probability for a Large Move in the S&P 500
We have now covered what market-implied probabilities are and how they are calculated and discussed their usefulness for policy makers.
But individual investors price risk differently based on their own situations and preferences. Because of this, it is helpful to strip off the market-implied costs that are baked into the risk-neutral probabilities. The real-world probabilities could then be used to weigh stress testing scenarios or evaluate the cost of other risk management techniques that align more with investor goals than option strategies.
Using the framework outlined above, we can go through an example of transforming the market implied probabilities of large moves in the S&P 500 into their real-world probabilities.
Statistical aside: The options data starts in 2007, and with 10 years of data, we only have 10 non-overlapping data points, which reduces the power of the maximum likelihood estimate used to fit the beta distribution. However, with options expiring twice a month, we have 24 separate data sets to use for calculating standard errors. Since we are concerned more about the potential differences between the risk-neutral and real-world distributions, we could use the rolling 12-month periods and still see the same trends. As with any analysis with overlapping periods, there can be significant autocorrelation to deal with. By using the 6-month distribution data from the Minneapolis Fed, we could double the number of observations.
Since the Minneapolis Fed calculates the market implied (risk neutral) probability distribution and the summary statistics (numerically), we must first translate it into a functional form to extend the analysis. Based on the data and the summary statistics, the distribution is neither normal nor log-normal. It is left-skewed and has fat tails most of the time.
Market Implied Probability Distributions for Moves in the S&P 500
Source: Minneapolis Federal Reserve
We will assume that the distribution can be parameterized using a skewed generalized t-distribution, which allows for these properties and also encompasses a variety of other distributions including the normal and t-distributions.[10] It has 5 parameters, which we will fit by matching the moments (mean, variance, skewness and kurtosis) of the distribution along with the 90th percentile value, since that tail of the distribution is generally the smaller of the two.[11]
We can check the fits using the reported median and the 10th percentile values to see how well they match.
Fit Percentile Values vs. Reported Values
Source: Minneapolis Fed. Calculations by Newfound Research.
There are instances where the reported distribution is bi-modal and would not be as accurately represented by the generalized skewed t distribution, but, as the above graph shows, the quantiles where our interest is focused line up decently well.
Now that we have our parameterized risk-neutral distribution for all time periods, the next step is to input the subsequent 12-month S&P 500 return into the CDF calculated at each point in time. While we don’t expect this risk-neutral distribution to necessarily produce a good forecast of the market return, this step produces the data needed to calibrate the beta function.
The graph below shows this CDF result over the rolling periods.
Cumulative Probabilities of Realized 12-month S&P 500 Returns using the Risk-Neutral Distribution from the Beginning of Each Period
Source: Minneapolis Fed and CSI. Calculations by Newfound Research.
The persistence of high and low values is evidence of the autocorrelation issue we discussed previously since the periods are overlapping.
The beta distribution function used to transition from the risk-neutral distribution to the real-world distribution has parameters j = 1.64 and k = 1.00 with standard errors of 0.09 and 0.05, respectively.
We can see how this function changes at the endpoints of the 95% confidence intervals for each parameter as a way to assess the uncertainty in the calibration.
Estimated Calibration Functions for 12-month S&P 500 Returns
Source: Minneapolis Fed and CSI. Calculations by Newfound Research. Data from Jan 2007 to Nov 2017.
When we transform the risk-neutral distribution into the real-world distribution, the calibration function values that are less than 1 in the left tail reduce the probabilities of large market losses.
In the right tail, the calibration estimates show that real-world probabilities could be higher or lower than the risk-neutral probabilities depending on the second parameter’s value in the beta distribution (this corresponds to k being either greater than or less than 1).
With the risk-neutral distribution and the calibrated beta distribution, we now have all the pieces to calculate the real-world distribution at any point in the options data set.
The graph below shows how these functions affect the risk-neutral probability density using the most recent option data. As expected, much more of the density is centered around the mode, and the distribution is skewed to the right, even using the bounds of the confidence intervals (CI) for the beta distribution parameters.
Risk Neutral and Real-World Probability Densities
Source: Minneapolis Fed and CSI. Calculations by Newfound Research. Data as of 11/15/17. Past performance is no guarantee of future results.
| Risk Neutral | Real-World | Real World (Range Based on Calibration) |
| Mean Return | 0.1% | 6.4% | 4.2% – 8.3% |
| Probability of a large loss (>20%) | 10.6% | 2.6% | 1.5% – 4.3% |
| Probability of a large gain (>20%) | 1.7% | 2.8% | 1.7% – 4.4% |
Source: Minneapolis Fed and CSI. Calculations by Newfound Research. Data as of 11/15/17. Past performance is no guarantee of future results.
Based on this analysis, we see some interesting things occurring.
- The mean return is considerably higher than the values suggested by many large firms, such as JP Morgan, BlackRock, and Research Affiliates.[12] Those estimates are generally for 7-10 years, so this doesn’t rule out having a good 2018, which is what the options market is showing.
- The real-world probability of a large loss is considerably lower than the 10.6% risk-neutral probability. When firms like GMO are indicating that 10% levels are already unreasonably low, their assessment of complacency in the market would only get stronger.[13]
- The lower end of the range for the real-world probability of a large gain is in line with the risk-neutral probability, suggesting that investors are seeking out risks (lower risk aversion) in a market with depressed yields on fixed income and low current volatility.
This also shows how looking at market implied probabilities can paint a skewed picture of the chances of an event occurring.
However, we must keep in mind that these real-world probabilities are still derived from the market-implied probabilities. In an efficient market world, all risks would correctly be priced into the market. But we know from the experience during the Financial Crisis that that is not always the case.
Our recommendation is to take all market probabilities with a grain of salt. Just because having a coin land on heads five times in a row has a probability of less than 4% doesn’t mean we should be surprised if it happens once. And coin flipping is something that we know the probability for.
Whether the market probabilities we use are risk-neutral or real-world, there are a lot of assumptions that go into calculating them, and the consequences of being wrong can have a large impact on portfolios. Risk management is important if the event occurs regardless of how likely it is to occur.
As with the weather, a 10% chance of a large loss versus a 4% chance is not a big difference in absolute terms, but a large portfolio loss is likely more devastating than getting rained on a bit should you decide not to bring an umbrella.
Conclusion
Market implied probabilities are risk-neutral probabilities derived from the derivatives market. If we assume that the market is efficient and that there is sufficient investor participation in these markets, then these probabilities can serve as a tool for governing organizations to adjust policy going forward.
However, these probabilities factor in both the actual probability of an event and the perceived cost to investors. Individual investors will attribute their own costs to such events (e.g. a retiree could be much more concerned about a 20% market drop than someone in the beginning of their career).
If individuals want to assess the probability of the event actually happening in order to make portfolio decisions, then they have to focus on the real-world probabilities. Ultimately, an investor’s cost function associated with market events depends more on life circumstances. While a bad state of the world for an investor can coincide with a bad state of the world for the market (e.g. losing a job when the market tanks), risk in an individual’s portfolio should be managed for the individual, not the “typical household”.
While the real-world probability of an event is typically dependent on an economic or statistical model, we have presented a way to translate the market implied probabilities into real-world probabilities.
With a better handle on the real-world probabilities, investors can make portfolio decisions that are in line with their own goals and risk tolerances.
[1] https://www.minneapolisfed.org/banking/mpd
[2] https://www.weather.gov/ffc/pop
[3] https://en.wikipedia.org/wiki/Put%E2%80%93call_parity
[4] https://www.minneapolisfed.org/~/media/files/banking/mpd/optimal_outlooks_dec22.pdf
[5] https://blog.thinknewfound.com/2015/03/weekly-commentary-folly-forecasting/
[6] https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2160157
[7] https://blog.thinknewfound.com/2017/09/the-lie-of-averages/
[8] https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2093397
[9] The beta distribution takes arguments between 0 and 1, inclusive, and has a non-decreasing CDF. It was also used in Fackler and King (1990) – https://www.jstor.org/stable/1243146.
[10] https://cran.r-project.org/web/packages/sgt/vignettes/sgt.pdf
[11] Since we have 5 unknown parameters, we have to add in this fifth constraint. We could also have used the 10th percentile value or the median. Whichever, we use, we can see how well the other two align with the reported values.
[12] https://interactive.researchaffiliates.com/asset-allocation/
[13] https://www.gmo.com/docs/default-source/research-and-commentary/strategies/asset-allocation/the-s-p-500-just-say-no.pdf
Are Market Implied Probabilities Useful?
By Nathan Faber
On November 27, 2017
In Risk & Style Premia, Weekly Commentary
This post is available as a PDF download here.
Summary
Market-implied probabilities are just as the name sounds: weights that the market is assigning an event based upon current prices of financial instruments. By deriving these probabilities, we can gain an understanding of the market’s aggregate forecast for certain events. Fortunately, the Federal Reserve Bank of Minneapolis provides a very nice tool for visualizing market-implied probabilities without us having to derive them.[1]
For example, say that I am concerned about inflation over the next 5 years. I can see how the probability of a large increase has been falling over time and how the probability of a large decrease has fallen recently, with both currently hovering around 15%.
Historical Market Implied Probabilities of Large Moves in Inflation
I can also look at the underlying probability distributions for these predictions, which are derived from the derivatives market, and compare the changes over time.
Market Implied Probability Distributions for Moves in Inflation
Source: Minneapolis Federal Reserve
From this example, we can judge that not only has the market’s implied inflation forecast increased, but the precision has also increased (i.e. lower standard deviation) and the probabilities have been skewed to the left with fatter tails (i.e. higher kurtosis).
Inflation is only one of many variables analyzed.
Also available is implied probability data for the S&P 500, short and long-term interest rates, major currencies versus the U.S. dollar, commodities (energy, metal, and agricultural), and a selection of the largest U.S. banks.
With all the recent talk about low volatility, the data for the S&P 500 over the next 12 months is likely to be particularly intriguing to investors and advisors alike.
Historical Market Implied Probabilities of Large Moves in the S&P 500
Source: Minneapolis Federal Reserve
The current market implied probabilities for both large increases and decreases (i.e. greater than a 20% move) are the lowest they have been since 2007.
Interpreting Market Implied Probabilities
A qualitative assessment of probability is generally difficult unless the difference is large. We can ask ourselves, for example, how we would react if the probability of a large loss jumped from 10% to 15%. We know that the latter case is riskier, but how does that translate into action?
The first step is understanding what the probability actually means.
Probability forecasts in weather are a good example of this necessity. Precipitation forecasts are a combination of forecaster certainty and coverage of the likely precipitation.[2] For example, if there is a 40% chance of rain, it could mean that the forecaster is 100% certain that it will rain in 40% of the area. Or it could mean that they are 40% certain that it will rain in 100% of the area. Or it could mean that they are 80% certain that it will rain in 50% of the area.
Once you know what the probability even represents, you can have a better grasp on whether you should carry an umbrella.
In the case of market implied probabilities, what we have is the risk-neutral probability. These are the probabilities of an event given that investors are risk neutral; these probabilities factor in the both the likelihood of an event and the cost in the given state of the world. These are not the real-world probabilities of the market moving by a given amount. In fact, they can change over time even if the real-world probabilities do not.
To illustrate these differences between a risk-neutral probability and a real-world probability, consider a simple coin flip game. The coin is flipped one time. If it lands on heads, you make $1, and if it lands on tails, you lose $1.
The coin is fair, so the probability for the coin flip is 50%. How much would you pay to play this game?
If you answer is nothing, then you are risk neutral, and the risk neutral probability is also 50%.
However, risk averse players would say, “you have to pay me to play that game.” In this case, the risk neutral probability of a tails is greater than 50% because of the downside that players ascribe to that event.
Now consider a scenario where a tails still loses $1, but a heads pays out $100. Chances are that even very risk-averse players would pay close to $1 to play this game.
In this case, the risk neutral probability of a heads would be much greater than 50%.
But in all cases, the actual likelihoods of heads and tails never changed; they still had a 50% real-world probability of occurring.
As with the game, investors who operate in the real world are generally risk averse. We pay premiums for insurance-like investments to protect in the states of the world we dread the most. As such, we would expect the risk-neutral probability of a “bad” event (e.g. the market down more than 20%) to be higher than the real-world probability.
Likewise, we would expect the risk-neutral probability of a “good” event (e.g. the market up more than 20%) to be lower than the real-world probability.
How Market Implied Probabilities Are Calculated
Note (or Warning): This section contains some calculus. If that is not of interest, feel free to skip to the next section; you won’t miss anything. For those interested, we will briefly cover how these probabilities are calculated to see what (or who), exactly, in the market implies them.
The options market contains call and put options over a wide array of strike prices and maturities. If we assume that the market is free from arbitrage, we can transform the price of put options into call options through put-call parity.[3]
In theory, if we knew the price of a call option for every strike price, we could calculate the risk-neutral probability distribution, fRN, as the second derivative with respect to the strike price.
where r is the risk-free rate, C is the price of a call option, K is the strike price and T-t is the time to option maturity.
Since options do not exist at every strike price, a curve is fit to the data to make it a continuous function that can be differentiated to yield the probability distribution.
Immediately, we see that the probabilities are set by the options market.
Are Market Implied Probabilities Useful?
Feldman et. al (2015), from the Minneapolis Fed, assert that market-based probabilities are a useful tool for policy makers.[4] Their argument centers around that fact that risk-neutral probabilities encapsulate both the probability of an event occurring – the real-world probability – and the cost/benefit of the event.
Assuming broad access to the options market, households or those acting on behalf of households can express their views on the chances of the event happening and the willingness to exchange cash flows in different states of the world by trading the appropriate options.
In the paper, the authors admit two main pitfalls:
However, they also refute many common arguments against using risk-neutral probabilities.
We have covered how policymakers often do not forecast very well themselves[5], which Ellison and Sargent argue may be intentional, stating that the FOMC may purposefully forecast incorrectly in order to form policy that is robust to model misspecification.[6]
Where a problem could arise is when an individual investor (i.e. a household) makes a decision for their own portfolio based on these risk-neutral probabilities.
We agree that having a financial market makes a “typical” investor more relevant than the “average fighter pilot” example in our previous commentary.[7] But what a central governing body uses to make decisions is different from what may be relevant to an individual.
The ability to be flexible is key. In this case, an investor can construct their own portfolio. It would be like a pilot constructing their own plane.
Getting to Real World Probabilities
Using the method outlined in Vincent-Humphreys and Noss (2012), we can transform risk-neutral probabilities into real-world probabilities, assuming that investor risk preferences are stable over time.[8]
Without getting too deep into the mathematical framework, the basic premise is that if we have a probability density function (PDF) for the risk-neutral probability, fRN, with a cumulative density function (CDF), FRN, we can multiply it by a calibration function, C, to obtain the real-world probability density function, fRW.
The beta distribution is a suitable choice for the calibration function.[9] Using a beta distribution balances being parsimonious – it only has two parameters – with flexibility, since it allows for preserving the risk-neutral probability distribution by simply shifting the mean and adjusting the variance, skew, and kurtosis.
The beta distribution parameters are calculated using the realized value that the market implied probability represents (e.g. change in the S&P 500, interest rates commodity prices, etc.) over the subsequent time period.
Deriving the Real-World Probability for a Large Move in the S&P 500
We have now covered what market-implied probabilities are and how they are calculated and discussed their usefulness for policy makers.
But individual investors price risk differently based on their own situations and preferences. Because of this, it is helpful to strip off the market-implied costs that are baked into the risk-neutral probabilities. The real-world probabilities could then be used to weigh stress testing scenarios or evaluate the cost of other risk management techniques that align more with investor goals than option strategies.
Using the framework outlined above, we can go through an example of transforming the market implied probabilities of large moves in the S&P 500 into their real-world probabilities.
Statistical aside: The options data starts in 2007, and with 10 years of data, we only have 10 non-overlapping data points, which reduces the power of the maximum likelihood estimate used to fit the beta distribution. However, with options expiring twice a month, we have 24 separate data sets to use for calculating standard errors. Since we are concerned more about the potential differences between the risk-neutral and real-world distributions, we could use the rolling 12-month periods and still see the same trends. As with any analysis with overlapping periods, there can be significant autocorrelation to deal with. By using the 6-month distribution data from the Minneapolis Fed, we could double the number of observations.
Since the Minneapolis Fed calculates the market implied (risk neutral) probability distribution and the summary statistics (numerically), we must first translate it into a functional form to extend the analysis. Based on the data and the summary statistics, the distribution is neither normal nor log-normal. It is left-skewed and has fat tails most of the time.
Market Implied Probability Distributions for Moves in the S&P 500
Source: Minneapolis Federal Reserve
We will assume that the distribution can be parameterized using a skewed generalized t-distribution, which allows for these properties and also encompasses a variety of other distributions including the normal and t-distributions.[10] It has 5 parameters, which we will fit by matching the moments (mean, variance, skewness and kurtosis) of the distribution along with the 90th percentile value, since that tail of the distribution is generally the smaller of the two.[11]
We can check the fits using the reported median and the 10th percentile values to see how well they match.
Fit Percentile Values vs. Reported Values
Source: Minneapolis Fed. Calculations by Newfound Research.
There are instances where the reported distribution is bi-modal and would not be as accurately represented by the generalized skewed t distribution, but, as the above graph shows, the quantiles where our interest is focused line up decently well.
Now that we have our parameterized risk-neutral distribution for all time periods, the next step is to input the subsequent 12-month S&P 500 return into the CDF calculated at each point in time. While we don’t expect this risk-neutral distribution to necessarily produce a good forecast of the market return, this step produces the data needed to calibrate the beta function.
The graph below shows this CDF result over the rolling periods.
Cumulative Probabilities of Realized 12-month S&P 500 Returns using the Risk-Neutral Distribution from the Beginning of Each Period
Source: Minneapolis Fed and CSI. Calculations by Newfound Research.
The persistence of high and low values is evidence of the autocorrelation issue we discussed previously since the periods are overlapping.
The beta distribution function used to transition from the risk-neutral distribution to the real-world distribution has parameters j = 1.64 and k = 1.00 with standard errors of 0.09 and 0.05, respectively.
We can see how this function changes at the endpoints of the 95% confidence intervals for each parameter as a way to assess the uncertainty in the calibration.
Estimated Calibration Functions for 12-month S&P 500 Returns
Source: Minneapolis Fed and CSI. Calculations by Newfound Research. Data from Jan 2007 to Nov 2017.
When we transform the risk-neutral distribution into the real-world distribution, the calibration function values that are less than 1 in the left tail reduce the probabilities of large market losses.
In the right tail, the calibration estimates show that real-world probabilities could be higher or lower than the risk-neutral probabilities depending on the second parameter’s value in the beta distribution (this corresponds to k being either greater than or less than 1).
With the risk-neutral distribution and the calibrated beta distribution, we now have all the pieces to calculate the real-world distribution at any point in the options data set.
The graph below shows how these functions affect the risk-neutral probability density using the most recent option data. As expected, much more of the density is centered around the mode, and the distribution is skewed to the right, even using the bounds of the confidence intervals (CI) for the beta distribution parameters.
Risk Neutral and Real-World Probability Densities
Source: Minneapolis Fed and CSI. Calculations by Newfound Research. Data as of 11/15/17. Past performance is no guarantee of future results.
Source: Minneapolis Fed and CSI. Calculations by Newfound Research. Data as of 11/15/17. Past performance is no guarantee of future results.
Based on this analysis, we see some interesting things occurring.
This also shows how looking at market implied probabilities can paint a skewed picture of the chances of an event occurring.
However, we must keep in mind that these real-world probabilities are still derived from the market-implied probabilities. In an efficient market world, all risks would correctly be priced into the market. But we know from the experience during the Financial Crisis that that is not always the case.
Our recommendation is to take all market probabilities with a grain of salt. Just because having a coin land on heads five times in a row has a probability of less than 4% doesn’t mean we should be surprised if it happens once. And coin flipping is something that we know the probability for.
Whether the market probabilities we use are risk-neutral or real-world, there are a lot of assumptions that go into calculating them, and the consequences of being wrong can have a large impact on portfolios. Risk management is important if the event occurs regardless of how likely it is to occur.
As with the weather, a 10% chance of a large loss versus a 4% chance is not a big difference in absolute terms, but a large portfolio loss is likely more devastating than getting rained on a bit should you decide not to bring an umbrella.
Conclusion
Market implied probabilities are risk-neutral probabilities derived from the derivatives market. If we assume that the market is efficient and that there is sufficient investor participation in these markets, then these probabilities can serve as a tool for governing organizations to adjust policy going forward.
However, these probabilities factor in both the actual probability of an event and the perceived cost to investors. Individual investors will attribute their own costs to such events (e.g. a retiree could be much more concerned about a 20% market drop than someone in the beginning of their career).
If individuals want to assess the probability of the event actually happening in order to make portfolio decisions, then they have to focus on the real-world probabilities. Ultimately, an investor’s cost function associated with market events depends more on life circumstances. While a bad state of the world for an investor can coincide with a bad state of the world for the market (e.g. losing a job when the market tanks), risk in an individual’s portfolio should be managed for the individual, not the “typical household”.
While the real-world probability of an event is typically dependent on an economic or statistical model, we have presented a way to translate the market implied probabilities into real-world probabilities.
With a better handle on the real-world probabilities, investors can make portfolio decisions that are in line with their own goals and risk tolerances.
[1] https://www.minneapolisfed.org/banking/mpd
[2] https://www.weather.gov/ffc/pop
[3] https://en.wikipedia.org/wiki/Put%E2%80%93call_parity
[4] https://www.minneapolisfed.org/~/media/files/banking/mpd/optimal_outlooks_dec22.pdf
[5] https://blog.thinknewfound.com/2015/03/weekly-commentary-folly-forecasting/
[6] https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2160157
[7] https://blog.thinknewfound.com/2017/09/the-lie-of-averages/
[8] https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2093397
[9] The beta distribution takes arguments between 0 and 1, inclusive, and has a non-decreasing CDF. It was also used in Fackler and King (1990) – https://www.jstor.org/stable/1243146.
[10] https://cran.r-project.org/web/packages/sgt/vignettes/sgt.pdf
[11] Since we have 5 unknown parameters, we have to add in this fifth constraint. We could also have used the 10th percentile value or the median. Whichever, we use, we can see how well the other two align with the reported values.
[12] https://interactive.researchaffiliates.com/asset-allocation/
[13] https://www.gmo.com/docs/default-source/research-and-commentary/strategies/asset-allocation/the-s-p-500-just-say-no.pdf