Nate Silver came into fame for his forecast of the presidential election in 2012. That matched his success in 2008 for also getting the presidential election correct. His book, The Signal and the Noise, is built upon that success.
We learn that Silver is not a political pundit, but a numbers geek. He started with statistical analysis of baseball while toiling away for the accounting firm KPMG. The next step was online poker, using his analysis to be a better poker player. That made him a lot of money, for a while. Then he turned to political analysis where he caught broader attention.
But the book is not a biography, or pages of Silver trumpeting his own horn. It’s a book about the success and failures of using data to forecast future events. Silver slices the data into two categories. The Signal is the truth and the Noise it what distracts us from the truth. In today’s market, information is not a scarce commodity. However, we perceive it selectively and subjectively. Our biggest failure is not paying attention to our own distortion. We never make perfectly objective predictions. They are always tainted by our subjective point of view.
Silver looks at the statistical analysis used in several fields:
- the housing bubble and mortgage securities crisis
- television political pundits
- baseball
- weather
- earthquakes
- economic forecasts
- flu epidemics
- poker
Taken from a scientific perspective, predictions can be tested. Silver tests them.
For example, look at weather forecasts. If the forecast calls for a 20% chance of rain, it should rain on 20% of the days that have that prediction. Silver backtests the performance of weather forecasters. He finds that some consistently over-forecast rain, raising a 5% chance to 20%. That way the audience will be less disappointed if it rains. Silver also compares long term forecasts to average conditions for that day. It turns out that the long term forecast is more likely to be wrong than merely relying on the historic average.
But weather forecasting has improved. Hurricane prediction is an area that has seen measurable improvement. Over the past 25 years the hurricane landfall prediction for three days in advance has decreased from a 350 mile radius to a 100 mile radius.
There is plenty in this book for compliance professionals. The first topic in The Signal and the Noise is the rating agencies and the financial crisis of 2008. He uses an example of S&P’s rating of CDO tranches. In achieving the AAA rating, S&P predicted that there was only a 0.12% probability that it would fail to pay out in the next five years. The reality is that the failure was 28%. The actual default rate was more than 200 times higher than predicted.