Jim Cullen: Review of Nate Silver's "The Signal and the Noise: Why So Many Predictions Fail -- But Some Don't" (Penguin, 2012)

Jim Cullen, who teaches at the Ethical Culture Fieldston School in New York, is a book review editor at HNN. His new book, Sensing the Past: Hollywood Stars and Historical Visions, was recently published by Oxford University Press. Cullen blogs at American History Now.

This one almost got away. Though I was a faithful reader of Nate Silver's 538 blog in The New York Times in the months running up to the presidential election -- like a lot of Democrats it gave me a serenity I otherwise would not have had -- the release of his first book fell through the cracks. I kept meaning to get around to it, partially put off by its length and my numbers phobia, though I bought a copy for my University of Chicago economics-majoring son (Silver is an alumnus of UC). When my son left it behind for me after a recent trip home, I finally got around to it. The Signal and the Noise was published and marketed as a book for the political season, which I'm sure made sense from a publicity standpoint. But it has a vitality and durability far beyond that. The paperback edition appears to be months away; when that moment arrives, I believe it will become a perennial.

As much as anything else the book is a study of epistemology, and a brief for the necessity and reality of uncertainty in everyday life. That's not because humans are flawed (though of course they are), or because Big Data has limits (it certainly does, as the book is at some pains to explain), but because our world is one of probabilities rather than fixed laws awaiting discovery and decoding. And yet for all the mistakes of recent years -- the failures to predict the financial crisis and 9/11 principal among them -- Silver argues it is possible to think about, and calculate, predictions that have utility in everyday life (poker, weather, athletic events) as well as in broader realms (financial markets, earthquakes, climate change). One can get better at predicting the way that one can get better at playing baseball: great hitters only make it to base a minority of the time, and yet compile a productive of record of consistency that's difficult for others to duplicate. The comparison is not incidental: Silver started his professional career by developing a system of forecasting player performance.

The heart of Signal is a popularization, of the ideas of Thomas Bayes (1701-1761), an English minister and mathematician who developed a formula of prediction that proved highly influential after his death, and which appears to be enjoying a renaissance. The wrinkle in Bayesian reasoning, and a source of ongoing controversy, is factoring in the perceived probability of an event happening before it ever does, a calculation that's hard to make and which is subject to change. For Silver, that's precisely the point: ongoing revision is the essence of the prognosticator's art. Statistics are the tools. He's not so much arguing for a particular methodology as he is a way of seeing the world. Which, given his serial successes in making a living playing cards, scouting ballplayers, and calling elections, hardly seems dreamy in its abstraction.

I'll cheerfully confess that much of what Silver does in this graph-rich book went over my head. But the various excursions he took me on -- in epidemiology, seismology, the real estate bubble, and the intricacies of gambling on the Los Angeles Lakers -- were so skillfully rendered, with so many colorful characters, that I never gave up in despair. His mind is both powerful and marvelous.