The Signal and the Noise: The Art and Science of Prediction, by Nate Silver

Review published on March 13, 2013.Reviewed by Stephen Joyce

Nudge Reviewer Rating:

[product sku=”9780141975658″]If there is one thing a history of prediction allows us to forecast with near certainty, it is that we will continue to spend billions of dollars being monumentally bad at prediction.

Consider that during the Cold War it seemed like half the Western world was devoted to analysing the behaviour of the Soviets, their intentions, their internal politics, their economic development, their foreign policy, their history and culture, their military, all were subject to massive sustained scrutiny for decades. And yet not one expert, despite all the information available, stood up on 1 Jan, 1989, and said: “You know what? I think the Berlin Wall might fall this year.” Most of them didn’t even predict it before the year 2050.

It’s because we’re all so notoriously bad at guessing the future that Nate Silver has shot to fame in recent years as a man who can, in fact, predict something. He has been consistently the most accurate predictor of American elections since 2008; in 2012, he correctly called the result of all 50 states, a feat that (despite the intense media scrutiny and relentless polling) virtually no one else managed to accomplish. In his interesting new book, The Signal and the Noise, Silver brings his expertise on the art and science of prediction to a variety of fields, from the Great Recession to baseball, from the online poker craze to earthquakes, from terrorism to weather forecasts.

Many of these individual chapters offer clear, thought-provoking insights into events of topical interest. In relation to the financial crisis of 2008, Silver notes that the ratings agencies branded junk mortgage bonds as AAA because they predicted that the failure of one mortgage would have no impact on any other mortgage, which completely ignored the idea of a housing bubble; they thus forecasted a default rate of 0.12% whereas the actual default rate was 28%. The most important lesson from this is that instead of expensive Harvard MBA graduates, ratings agencies would be better off shaving some chimps and giving them a blindfold and a set of darts.

Silver has previous experience in baseball, having been part of the Moneyball phenomenon. One of his techniques was to compare a young player’s stats against similar players in history at the same age and then predict from the historical players’ careers how good the player would become. This had some big hits, but also bad misses. As Silver notes, the most successful teams are those which combine close attention to stats and good scouting, which evaluates things like personality and temperament that numbers often miss.

In poker, which Silver played professionally enough to pay his way through university, we find out that it is a case of trickle-up economics – the worst player is losing enough to support at least half the table. But poker players may make good predictions about an opponent’s hand and still get rocked by an unlikely card. You only win in the long run; in the short run, which may go for a whole year, you may play correctly and still lose to the shaved chimp or, worse, the Harvard MBA.

The chapters can feel rather discrete, but a unifying theme emerges in Ch. 8, when Silver introduces his statistical hero, Thomas Bayes, an English minister born in 1701. Bayes’ philosophy emphasised that human beings can never have perfect knowledge; we need to think in terms of how likely something is to be true. His theorem for doing so is relatively simple and simply means getting closer and closer to the truth with an ongoing series of experiments. For Silver:

Science may have stumbled later when a different statistical paradigm, which deemphasised the role of prediction and tried to recast uncertainty as resulting from the errors of our measurements rather than the imperfection of our judgments, came to dominate in the twentieth century.

Readers of philosophy will recognise this as the statistical equivalent of American pragmatism, as expounded by William James. I’m an enormous fan of James’ brilliant writings and the pragmatic method, but it does have a weakness that Silver cannot overcome.

Bayes’ theorem works like this: you have a prior probability (how likely you think something is to happen), then you get some new information which causes you to revise your original idea to produce a posterior probability, which is then the basis for new calculations. The problem is: where do you get the original prior probability? Like pragmatism, it works fine once you have something to work with, but the starting point is always obscure.

The difficulties become clear when Silver writes about the 9/11 terrorist attacks. On the morning of 9/11, people would have said the prior probability of such an event happening was minuscule, but they could only know they were wrong after it happened. Bayes’ theorem depends on time for an extended or infinite series of similar events to occur, so it works well for examining poker games or market trades or political opinion polls before an election, but it won’t work for things that strike out of the blue, even though those things are likely to be the most devastating.

Silver is aware of this, but he doesn’t have a solution. That’s not his fault; no one does. But it indicates why predictions will continue to fail despite improvements in computing power because computers cannot provide the initial assumptions on which the calculations are based, and the initial assumptions are often just pure guesswork.

The book does stress the importance of acknowledging uncertainty in prediction, which is a good thing. The most confident predictors are usually the most wrong; the people who acknowledge the limitations of their models are very often right. They just make for bad media pundits because we prefer opinionated buffoons declaring they can see perfectly through their blindfolds as they step confidently over the cliff.

Overall, it’s an interesting read, but I’m not sure its big idea is really big enough. The book amounts primarily to case studies of prediction across a variety of fields, which is perhaps more valuable than one major insight, but the flipside is that it lacks overall coherence. Anyone hoping the book will give them the inside track on predicting the future will be somewhat disappointed. The book won’t make you more right; it may make you less wrong. That is a step in the right direction, but only a step.

Previous:

An extract from The God Argument, by A C Grayling

Next:

Elliot Perlman on The Street Sweeper

You may also like