Tuned In

Moneypoll! The Pundits Vs. The Election-Data Nerds

The other big contest next Tuesday is between the pundits trying to analyze the election with their guts and a new breed of statistics gurus trying to forecast it with data.

  • Share
  • Read Later
Joe Scarborough, Host, MSNBC’s “Morning Joe” appears on "Meet the Press" in Washington D.C., Sunday, Sept. 23, 2012.
William B. Plowman / NBC / NBC NewsWire

Joe Scarborough, Host, MSNBC’s “Morning Joe” appears on "Meet the Press" in Washington D.C., Sunday, Sept. 23, 2012.

Come next Tuesday night, we’ll get a resolution (let’s hope) to a great ongoing battle of 2012: not just the Presidential election between Barack Obama and Mitt Romney, but the one between the pundits trying to analyze that race with their guts and a new breed of statistics gurus trying to forecast it with data.

In Election 2012 as seen by the pundits–political journalists on the trail, commentators in cable-news studios–the campaign is a jump ball. There’s a slight lead for Mitt Romney in national polls and slight leads for Barack Obama in swing-state polls, and no good way of predicting next Tuesday’s outcome beyond flipping a coin.

In Election 2012 as seen by the stats guys–there are many, though Nate Silver of the New York Times’s Fivethirtyeight blog has been getting most of the attention–the campaign is not a lock, but Obama has a clear advantage. By averaging out the results of state and national polls, considering the accuracy and trends in those polls over past years and figuring in economic and historical factors (or not), the data guys have generally (though not universally) given the President a clear edge: currently 77% in Silver’s model, north of 90% at the Princeton Election Consortium. (Their reasoning boils down to the President’s small but consistent lead in enough battleground state polls to win the Electoral College, plus the historical record of state vs. national polls. I’m oversimplifying here, though.)

Can you call an election by applying math rather than going to rallies and talking to cab drivers and diner customers? As a political-news junkie, I’ve been noticing some passive-aggressive sniping against the data guys by more traditional reporters for a while now. Howard Fineman, for instance, recently tweeted that the Des Moines Register endorsement of Romney made clear for the first time that Obama might lose—though “it’s not scientific or quantifiable by Nate Silver.” Meow!

And in a Politico article by Dylan Byers Monday, that low-simmering catfight became an all out cat war. Byers suggested that Silver could become “a one-term celebrity” if Obama loses, and quoted media stars like Joe Scarborough scoffing at Professor Science’s efforts to quantify something that, to them, can’t be measured in a spreadsheet. “Nate Silver says this is a 73.6 percent chance that the president is going to win?” Scarborough said. “Nobody in that campaign thinks they have a 73 percent chance — they think they have a 50.1 percent chance of winning.” Silver’s statistics? A “joke,” said Scarborough.

You can read politics into this dispute. It’s undeniable that Silver has become a kind of polestar—or poll star—for nervous liberals, as his rating of Obama’s chances have never dipped below 60% in the general election. Some conservatives, meanwhile—many of whom have criticized the polls as “skewed” toward Democrats this whole election—have insinuated that Silver, once a commentator at Daily Kos, has his thumb on the scales for liberals. (The counter to this being that, politics aside, results are results, and Silver’s have been very strong—if only, at this point, for a few years.)

But there’s also a kind of territorial, John Henry vs. the Steam Drill, Kasparov vs. Deep Blue, man-vs.machine tone of defensiveness here—a sense that the Hari Seldon–like attempt to quantify mass human reaction through numbers rather than intuition is fruitless, if not just flat-out wrong.

And it’s a familiar one, especially if you’re a sports fan, or at least a movie fan like me. When I tweeted about this phenomenon the other day, fellow TV critic—and much bigger baseball fan than I—Alan Sepinwall noted that this is the exact same kind of reaction that sabermetric baseball analysts got from old-school coaches and scouts, as chronicled in the Michael Lewis book and movie Moneyball.

This is no coincidence. Before Silver was calling elections, he was a leading sabermetrics analyst, writing for Baseball Prospectus. As Moneyball showed us, the idea of choosing players on the basis of number crunching generated a pushback that was truly visceral—in the sense of defending your gut over someone else’s brain. Baseball isn’t calculus! Your computer can’t tell you whether a player has a spark! Likewise, the reaction against Nate Silver and company rings of defending experience, show leather and motherwit against algorithms. Your computer can’t feel the public mood shifting in Ames, Iowa!

The baseball scouts threatened by Bill James and his disciples were at least defending a way of life. Their job was simply to identify and place a value on baseball players. The skepticism toward poll analysts comes from political reporters whose jobs have become increasingly all about the horse race—telling us who’s “winning” and “losing” and why—but don’t necessarily need to.

Really, there’s no reason that analysts like Nate Silver—should their forecasts prove more reliable over time—need to “replace” political journalists. Instead, they could free up political reporters to things that are more useful for all of us: reporting on the differences between candidates’ policies, about what government does and doesn’t do, and about how elections can address our problems. If pundits could offload the quantification to the quants, it would be much better all around for the news audience.

Instead, though, it looks like some pundits are looking for an opportunity to delegitimize the poll aggregators and analysts en masse—especially if they give Obama better odds and he loses.

Unfortunately, that would depend on a misunderstanding of odds that their audience may not appreciate. If you say Obama has a three-in-four chances of winning, it means that Romney has a one-on-four chance: still quite a decent shot. But herein lies the difference between using these metrics in presidential elections vs. baseball. In a baseball season, you get 162 chances to be right or wrong—the accuracy or inaccuracy of your model can be tested over a number of results. When you have one Presidential election every four years, in the public mind–unfairly, but still–your reputation for being “right” or “wrong” will be grossly overinflated on the basis of a relatively few results. (Also, Silver et al. are only as good as their data—they’re not pollsters themselves and so are as vulnerable as any prognosticator if polls are wrong en masse.)

There is where the traditional pundits have some advantages, ones that have protected them and their reputations since time immemorial. As we’ve seen with one political development after another, it’s a rare pundit whose job security has been threatened by being wrong about anything—especially if they manage to be wrong in the same way that most of their colleagues are.

And maybe more important for the pundits: the beauty of calling an election a 50/50 tossup is that 100% of the time, you will be right.

23 comments
StopidHunnicutt
StopidHunnicutt

Again - these are data GEEKS.  A geek is someone who is immersed or obsessed with something of a technical nature; a nerd is someone who can't get a date for the prom.  Please use accordingly.

drbob211
drbob211

While the popular vote is close, Princeton Election Consortium uses state polling data ( a meta-analysis) to predict election outcome.  As of 11/2/12 noon, they give odds of an Obama victory at 99%.  BTW, UK betting is 66% Obama., 539 is 79% Obama.

Bemused
Bemused

The idea of a pundit saying Silver should be legitimized if his forecast is wrong this time is laughable. Pundits suffer no repercussions for consistently being wrong--in fact, they're more likely to get booked if they make bold, eye-catching predictions, right or wrong. The very people casting these aspersions against silver are the most Teflon and often useless professionals out there.

A study published last year looked at 26 of the usual Sunday morning talking heads and columnist. Over a 16-month period, the researchers "found that only nine of the prognosticators they studied could predict more accurately than a coin flip. Two were significantly less accurate, and the remaining 14 were not statistically any better or worse than a coin flip." (http://www.hamilton.edu/news/story/pundits-as-accurate-as-coin-toss-according-to-study)

RobertSalter
RobertSalter

Not only do pundits fear loss or redirection of business (as Time notes) but they, like physicians past, have a deep visceral dislike for applying the (social) scientific to their realm of intuitive knowledge.

Also, most have a fundamental lack of understanding of how statistics work and more importantly what they mean--how we should feel about them, evaluate them, consider them.

Bemused
Bemused

"Instead, they could free up political reporters to things that are more useful for all of us: reporting on the differences between candidates’ policies, about what government does and doesn’t do, and about how elections can address our problems." Ha--good one, James! I'd like to get Nate Silver's odds on that happening. After all, there's nothing stopping reporters from doing that now.

buzzerj
buzzerj

I give Nate Silver a 68.43% chance of being correct.

TomShaw
TomShaw

In my opinion, the traditional pundits' dislike of metrics guys like Silver isn't just because he may be right on election night, but that he could be right for the entire electoral news cycle (nearly half the presidential term). Each national poll that shows a significant swing from the last national poll provides days of easy news filler (and the poll itself is relatively cheap to produce), but an electoral, state-by-state level poll is both less likely to show a significant swing and far more difficult, and expensive, to produce. 

If the Silvers of the world are right, much of the pundits' year-round production will be rendered moot.

Disclaimer 1: Some of my favorite websites are Football Outsiders, TV By The Numbers, and Grantland (Barnwell division). So I am already in the metrics' pocket.

Disclaimer 2: I voted for McCain, I voted for Romney. I think there is practically no chance Romney wins the electoral vote (he takes maybe two toss-up states), and only a tiny chance he wins the general vote.

dcchrismich
dcchrismich

Well, given that the whole "Moneyball" story has proven a demonstrably false - though a very seductive and very lucrative, pseudo-quantitative - myth...  What does that say?  Citing "Moneyball" undermines one's arguments; it does not support them.

KennyJohnson
KennyJohnson

"When you have one Presidential election every four years, in the public mind–unfairly, but still–your reputation for being “right” or “wrong” will be grossly overinflated on the basis of a relatively few results."

Not exactly. Nate's model can still be tested because he's not only giving Obama at 77% chance of winning the EC, he's giving probabilities for every state and showing the potential vote breakdown (e.g. Florida: O 49.3 / R 50.1). So if Obama wins Florida 50.1 with Romney getting 49.3 does it mean Nate's model is wrong? No, it means he was incredibly close! 

However if Romney wins Florida and Ohio and Colorado with 10 pts each, then Nate's model (or the polls) is seriously flawed. 

wandmdave
wandmdave

"Really, there’s no reason that analysts like Nate Silver—should their forecasts prove more reliable over time—need to “replace” political journalists. Instead, they could free up political reporters to things that are more useful for all of us: reporting on the differences between candidates’ policies, about what government does and doesn’t do, and about how elections can address our problems."But that takes time, money, and the balls to let go of relativism and call even entrenched opinions right or wrong based on observation. I don't see that happening even if the stats guys wipe the floor with the windbags (as I'm sure they will as a whole over time, haruspicy isn't all that great at predicting the future it turns out).

anon76
anon76

Slight correction, James:  Nate has actually been pretty good about pointing out metrics that can be applied beyond just "did they call the correct winner".  For instance, if Nate picks Obama to win Colorado by 0.2%, but in fact Romney wins by 0.1%, Nate will be much happier than if he predicts Obama to win California by 32%, and Obama in fact wins it by 12%.  In the former case, Nate will have been fairly accurate, even if he called the wrong winner, while in the latter case the opposite will be true.  Unfortunately for Nate, the pundits don't seem to understand this nuance, and will simply crow if he calls the wrong winner (ignoring whether or not the result is within the 95% confidence interval that he provides 95% with his predictions).

With your caveat about "only as good as the available data" in mind, I am supremely confident that the nerds are going to wipe the floor with the bloviators in six days.  Unfortunately, if I've learned anything from Bill Kristol's career path, it's that being wrong about everything can actually be net positive for a pundit.

dmbfan93933
dmbfan93933

I've been reading Nate's blog for awhile now and do not see that he is biased concerning the results of the election for either candidate.  I don't fault him for not sharing all the assumptions he has built into his model.  It's his work after all. 

It wouldn't make any sense for Nate to "cook the books" for either candidate as Nate's reputation is built upon, and will be judged by the accuracy of his model. 

pentadactyl
pentadactyl

They're all basically predicting the same thing.  They'll all claim victory and go home feeling superior to the other.  The only ones who would have to eat their hats are the ones claiming a blowout for either side (a chance Nate Silver puts at 0.5 and <0.1 percentage).  But even they won't admit defeat as anyone making such predictions is already way down the rabbit hole. 

charlieromeobravo
charlieromeobravo

"There’s a slight lead for Mitt Romney in national polls and slight leads for Barack Obama in swing-state polls, and no good way of predicting next Tuesday’s outcome beyond flipping a coin."

Until they abolish the electoral college system, this is a completely B S statement.  At this late a date, national polls are fun to talk about but they have no real meaning.

Pundits spin for their respective sides.  You can say the same about the quants but at least they have numbers to back up their positions.  Ignore the quants that won't show their work, pay attention to the ones that do and how they do it.  I'll happily pay more attention to predictions from the quants with legitimate models over predictions from knobs like Paul Begala or Bill Krystol.  

Silver is taking some heat lately because his model has been consistently predicting an Obama win.  In the GOP's war against facts and objectivity, there is nothing that isn't questionable if it isn't going their way.  But Nate is very transparent about how his model is constructed and how it arrives at its conclusions.  His model is objective, his critics are not.  They point to his admission that he leans liberal socially as evidence of his bias but no one has been able to mount a criticism of his model.  

All of the shots fired at him are just the last desperate days of the Romney campaign trying to fight any message that runs counter to their "we're winning" narrative.  

PaulDirks
PaulDirks

Considering the track record of pundits who've made themselves famous talking to cabdrivers, I'd say assigning the advantage to the number-crunchers is a severe no-brainer. Thanks for covering this vitally important story!

Bemused
Bemused

That should be "de-legitimized." I obviously miss the editing function!

arias
arias

 @dcchrismich If what you got out of Moneyball was that the methodology from the sabermetric number crunchers were "demonstrably false" then sorry but you got the narrative all wrong. Sabermetrics is deployed by every major league baseball team today, each has their saber specialist because of the revolution that Bill James and Billy Beane started that Nate was quite integral to. If you're saying it's "demonstrably false" because the A's never experienced sustained winning and never won a world series, then your knowledge of the facts are sorely lacking. What happened is that ALL the teams began deploying sabermetrics to catch up with the competitive advantage that Beane had, and this eliminated his competitive advantage. But the success of the movement could not be any more clear by the types of statistics teams value most these days. Where OBP is superior to batting average and OPS (OBP + Slugging %) reigns supreme.  

Bemused
Bemused

@dcchrismich I don't he cited Moneyball to show that Silver's methods are superiors but to illustrate how old guards react to new-fangled ways of doing things (e.g., gut vs. stats).

anon76
anon76

Second slight correction, James- Nate is actually using a slightly wider universe of polls to get his national average- polls such as yougov.com, google consumer surveys, and surveys from the RAND corporation, that aren't widely cited by pundits nor aggregated by sites such as real clear politics.  These polls have been more favorable towards Obama than the much more widely cited Rassmussen and Gallup polls, and as a result Nate is showing Obama with an advantage in the national polls without having to take into account the state polls.  Granted, even the national poll lead that Nate is showing for Obama is less than the state polls, but it would be wrong to say that simple averages of state vs. national polls show opposite results, at least not if you use the same data set that Nate Silver does.

poniewozik
poniewozik

@charlieromeobravo In defense of the "BS statement" -- not that I subscribe to it but to play devil's advocate -- the argument is, partly, that while the electoral college and popular vote CAN spilt, historically they rarely do, and when they do it tends to be with a very slim popular vote margin. Ergo (by this way of thinking) history suggests that either the state polls or the national ones are right, but we don't know which. 

(IIRC Nate Silver has been making a form of this argument too--it's just that, in the view of the quants, the state polls generally have history, and a greater combined sample size, on their side. Thus Silver's model also calls for an Obama popular vote win. But I risk wading out beyond my knowledge of both politics and statistics here.)

Also, I would note that there ARE criticisms of Silver's model out there, even among some of the quants. (E.g., that he does not reveal what his model is 100%.) That's to be expected--different modelers will disagree, and their records will have to bear them out. But those are different from "he's cooking the numbers for Barry."

Lucelucy
Lucelucy

@PaulDirks Some of my best friends have been cab drivers.  None of my best friends have been pundits.  Well, not on TV, anyway.  ♫

anon76
anon76

James- I'm surprised to hear that other quants have complained about Nate's model.  He claims that transparency is pretty important to him, and, having followed him pretty closely for over 4 years now, I'm confident I could write code to re-create his results.  It's true that he doesn't just come write out and print the formulas (as does, for instance, www.electionprojection.com or www.electoral-vote.com), but his model involves more complicated math than simply addition, subtraction, and division.  However, as far as I'm concerned he has provided enough methodology for others to recreate his results (the scientific nerd threshold, if you will).