The Signal and the Noise: Why So Many Predictions Fail-but Some Don't Spiral-Bound | September 27, 2012
Nate Silver
★★★☆☆+ from 10,001 to 50,000 ratings
The Signal and the Noise: Why So Many Predictions Fail-but Some Don't
UPDATED FOR 2020 WITH A NEW PREFACE BY NATE SILVER
"One of the more momentous books of the decade." —The New York Times Book Review
Nate Silver built an innovative system for predicting baseball performance, predicted the 2008 election within a hair’s breadth, and became a national sensation as a blogger—all by the time he was thirty. He solidified his standing as the nation's foremost political forecaster with his near perfect prediction of the 2012 election. Silver is the founder and editor in chief of the website FiveThirtyEight.
Drawing on his own groundbreaking work, Silver examines the world of prediction, investigating how we can distinguish a true signal from a universe of noisy data. Most predictions fail, often at great cost to society, because most of us have a poor understanding of probability and uncertainty. Both experts and laypeople mistake more confident predictions for more accurate ones. But overconfidence is often the reason for failure. If our appreciation of uncertainty improves, our predictions can get better too. This is the “prediction paradox”: The more humility we have about our ability to make predictions, the more successful we can be in planning for the future.
In keeping with his own aim to seek truth from data, Silver visits the most successful forecasters in a range of areas, from hurricanes to baseball to global pandemics, from the poker table to the stock market, from Capitol Hill to the NBA. He explains and evaluates how these forecasters think and what bonds they share. What lies behind their success? Are they good—or just lucky? What patterns have they unraveled? And are their forecasts really right? He explores unanticipated commonalities and exposes unexpected juxtapositions. And sometimes, it is not so much how good a prediction is in an absolute sense that matters but how good it is relative to the competition. In other cases, prediction is still a very rudimentary—and dangerous—science.
Silver observes that the most accurate forecasters tend to have a superior command of probability, and they tend to be both humble and hardworking. They distinguish the predictable from the unpredictable, and they notice a thousand little details that lead them closer to the truth. Because of their appreciation of probability, they can distinguish the signal from the noise.
With everything from the health of the global economy to our ability to fight terrorism dependent on the quality of our predictions, Nate Silver’s insights are an essential read.
“Mr. Silver, just 34, is an expert at finding signal in noise… Lively prose — from energetic to outraged… illustrates his dos and don’ts through a series of interesting essays that examine how predictions are made in fields including chess, baseball, weather forecasting, earthquake analysis and politics… [the] chapter on global warming is one of the most objective and honest analyses I’ve seen… even the noise makes for a good read.”
—New York Times
“Not so different in spirit from the way public intellectuals like John Kenneth Galbraith once shaped discussions of economic policy and public figures like Walter Cronkite helped sway opinion on the Vietnam War…could turn out to be one of the more momentous books of the decade.”
—New York Times Book Review
"A serious treatise about the craft of prediction—without academic mathematics—cheerily aimed at lay readers. Silver's coverage is polymathic, ranging from poker and earthquakes to climate change and terrorism."
—New York Review of Books
"Mr. Silver's breezy style makes even the most difficult statistical material accessible. What is more, his arguments and examples are painstakingly researched..."
—Wall Street Journal
"Nate Silver is the Kurt Cobain of statistics... His ambitious new book, The Signal and the Noise, is a practical handbook and a philosophical manifesto in one, following the theme of prediction through a series of case studies ranging from hurricane tracking to professional poker to counterterrorism. It will be a supremely valuable resource for anyone who wants to make good guesses about the future, or who wants to assess the guesses made by others. In other words, everyone."
—The Boston Globe
"Silver delivers an improbably breezy read on what is essentially a primer on making predictions."
—Washington Post
“The Signal and the Noise is many things — an introduction to the Bayesian theory of probability, a meditation on luck and character, a commentary on poker's insights into life — but it's most important function is its most basic and absolutely necessary one right now: a guide to detecting and avoiding bullshit dressed up as data…What is most refreshing… is its humility. Sometimes we have to deal with not knowing, and we need somebody to tell us that.”
—Esquire
“[An] entertaining popularization of a subject that scares many people off… Silver’s journey from consulting to baseball analytics to professional poker to political prognosticating is very much that of a restless and curious mind. And this, more than number-crunching, is where real forecasting prowess comes from.”
—Slate
“Nate Silver serves as a sort of Zen master to American election-watchers… In the spirit of Nassim Nicholas Taleb’s widely read “The Black Swan”, Mr. Silver asserts that humans are overconfident in their predictive abilities, that they struggle to think in probabilistic terms and build models that do not allow for uncertainty.”
—The Economist
"Silver explores our attempts at forecasting stocks, storms, sports, and anything else not set in stone."
—Wired
"The Signal and the Noise is essential reading in the era of Big Data that touches every business, every sports event, and every policymaker."
—Forbes.com
“Laser sharp. Surprisingly, statistics in Silver’s hands is not without some fun.”
—Smithsonian Magazine
“A substantial, wide-ranging, and potentially important gauntlet of probabilistic thinking based on actual data thrown at the feet of a culture determined to sweep away silly liberal notions like ‘facts.’”
—The Village Voice
“Silver shines a light on 600 years of human intelligence-gathering—from the advent of the printing press all the way through the Industrial Revolution and up to the current day—and he finds that it's been an inspiring climb. We've learned so much, and we still have so much left to learn.”
—MLB.com
“Nate Silver’s The Signal and the Noise is The Soul of a New Machine for the 21st century (a century we thought we’d be a lot better at predicting than we actually are). Our political discourse is already better informed and more data-driven because of Nate’s influence. But here he shows us what he has always been able to see in the numbers—the heart and the ethical imperative of getting the quantitative questions right. A wonderful read—totally engrossing.”
—Rachel Maddow, author of Drift
“Yogi Berra was right: ‘forecasting is hard, especially about the future.’ In this important book, Nate Silver explains why the performance of experts varies from prescient to useless and why we must plan for the unexpected. Must reading for anyone who cares about what might happen next.”
—Richard Thaler, co-author of Nudge
“Making predictions in the era of ‘big data’ is not what you might imagine. Nate Silver's refreshing and original book provides unpredictably illuminating insights differentiating objective and subjective realities in forecasting our future. He reminds us that the human element is still essential in predicting advances in science, technology and even politics... if we were only wise enough to learn from our mistakes.”
—Governor Jon Huntsman
“Here's a prediction: after you read The Signal and the Noise, you'll have much more insight into why some models work well—and also why many don't. You'll learn to pay more attention to weather forecasts for the coming week—and none at all for weather forecasts beyond that. Nate Silver takes a complex, difficult subject and makes it fun, interesting, and relevant.”
—Peter Orszag, former director of the Office of Management and Budget
“Projection, prediction, assumption, trepidation, anticipation, expectation, estimation… we wouldn’t have 80 words like this in the English language if it wasn’t central to our lives. We tend not to take prediction seriously because, on some level, we know that we don’t know. Silver shows us how this inevitable part of life goes awry when projected on a grand scale into the murky worlds of politics, science and economics. Dancing through chess, sports, snowstorms, global warming and the McLaughlin Group, he makes a serious and systematic effort to show us how to clean the noise off the signal.”
—Bill James, author of The Bill James Baseball Abstracts