Block That Inference

George Stephanopolous reads exit polls:

We did ask a question I know in the exit polls about Reverend Wright, Barack Obama’s former pastor and whether that was influencing voters. What did we find? Right down the middle. About half said it’s important, about half said it was unimportant. Of those who said it was important, look at this in Indiana, 70% went for Senator Clinton. Of those who thought it was unimportant, again right down the middle, 65% for Barack Obama. So what you thought about the importance of Reverend Wright basically determined your vote.

As Lee Siegelman points out the causal inference here is all wrong. Much more likely is that voters already committed to Clinton -- or strongly predisposed to commit to Clinton -- adopted the view that Reverend Wright was an important issue because they knew it was an issue that reflected poorly on Obama. Note, for example, that both pre- and post-Wright, both Clinton and Obama took a fairly constant share of different demographic categories.

The thing is, you shouldn't need to be especially sophisticated about statistics to figure this out. Clearly, Wright may have swayed some voters, but equally clearly most people had opinions about the election before Wright ever came on the scene. But Stephanopolous is hardly alone here, almost every time I see exit polls discussed on TV it's done with almost no understanding of how to read them properly.

Comments

If every pundit who foolishly equated correlation with causation was hung from the nearest lamp post, we'd have a huge price spike for ropes.

Posted by Will Allen | May 12, 2008 3:05 PM

And it would be worth every penny.

Posted by Steve LaBonne | May 12, 2008 3:08 PM

And, as you can see, an increase in rope prices resulted in a consequent spike in street lamp failure.

Posted by Tyro | May 12, 2008 3:11 PM

. . . But in that case, it would indeed be causation. Would kinda serve 'em right.

Posted by maxgowan | May 12, 2008 3:13 PM

The airheads have to read the statistics that way so that they can justify having flogged the issue to death.

Posted by Jeffrey Davis | May 12, 2008 3:26 PM

And NO ONE ever gets called for this. Its insane.

Posted by Leo | May 12, 2008 3:28 PM

"As Lee Siegelman points out the causal inference here is all wrong."

Several cable news folks made the same idiotic conclusion.

That exit poll should have read, “Please list the following pretext for your vote.”

Posted by Christopher Colaninno | May 12, 2008 3:28 PM

I remember thinking basically the same thing after Pennsylvania. The narrative leading up to Ohio and Texas was that Hillary had a three state "firewall" that would blunt Obama's momentum and it was up to him to make inroads there. Then, in the six weeks between OH&TX and PA, the Wright story (among other things) broke against Obama and the fact he lost PA--by less than was predicted and which he was always supposed to lose anyway!--was proof of Wright's resonance as an issue. And of course now it's all rolled into the common wisdom and all pre-Wright narratives are lost to the ether.

Posted by mark f | May 12, 2008 3:30 PM

Stephonopolous is not an idiot. The probability that he is unaware of the idiocy of his statement is in the single digits. He's just selling the story. When a politician lies, it's often an unprincipled means to advance a principle sincerely held. This is something altogether more disgusting.

Posted by Led | May 12, 2008 3:33 PM

A similar blunder: Exit data says the Republican vote in Indiana was split. Some pundits think that implies Limbaugh's Operation Chaos had no effect. You can't measure a delta with one data set.

Posted by matt | May 12, 2008 3:34 PM

Yes! Thank you for posting this. It is often frustrating to watch any "analysis" of exit polls -- generally, the talking heads just use the raw statistics to further the campaign narratives they have already established. For the past several primary contests, I've watched a variety of television personalities attribute Clinton's success among late-deciding voters to whatever negative story was then plaguing Obama. So the fact that late-deciders in Pennsylvania favored Clinton was attributed to Wright and when it happened in Ohio/Texas it was attributed to Clinton going negative. At no point did any pollster point out that the late-deciders have favored Clinton throughout the primaries, even in states Obama carried. There is a plausible argument to be made that Clinton does well among these voters because she is so well known -- if you are indecisive, you're more likely to go with the name brand. This is an important story -- if Obama hopes to beat McCain, he has to find a way to overcome the fact that voters are far more familiar with McCain, and familiarity breeds comfort. But it's been largely ignored in terms of analysis because it does not offer an opportunity to play a "God damn America!" snippet.

Also notable: Stephanopolous and ABC can use these exit poll results to continue to justify the way they handled the Pennsylvania debate. Bonus!

Posted by Emily | May 12, 2008 3:38 PM

I agree with Led. Steph knows exactly what he's doing. It's not just about selling the story, it's about self-justification for the way he handled the story.

Posted by blah | May 12, 2008 3:38 PM

I'm going to disagree here; I think Stephenopoulus is an idiot and actually does think everything is about Hillary. That is, it's self-justification for the way he handled the story, but he believes it shows that he is a serious journalist; he's not so much Rovian as Bushian.

Posted by Deborah | May 12, 2008 3:48 PM

I think that George is smart enough to know that he was in error, that he misspoke. I'll leave it to others to divine his mental processes leading to the slip.
Isn't it more likely that George meant to say that one's take on Wright/Obama was indicative not determinative of one's vote?
It is after all highly unlikely that George thinks any one issue, even Obama's serial lies about Wright, determined any voter's choice.
This sort of careless phrasing is part and parcel of our entire public discourse, so much so that the examples one chooses to highlight are surely indicative of one's prejudices.
I think more people need to regularly go on tv and before a large audience so as to realize just how easy it is to misspeak. I am amazed that Matt with his almost daily reminders of his own written and spoken carelessness is so little inclined to cut others the same slack he wants for himself.
Then again I'm not really surprised; as the posts make clear this is somehow to be taken as a challenge to the Obama candidacy. The threat of four years of this sort of ObaIdiocy is really all we need to insure his GE defeat.
So thanks!

Posted by JTHB | May 12, 2008 4:08 PM

So true. TV commentary is full of misreadings of statistics and poll results. If a poll said that 94% of women like chocolate ice cream, while only 85% of men do, it would take no more than an hour for someone to say, "Women like chocolate more than men do" (wrong) and only a day or so for someone to say, "Men don't like chocolate!"

Posted by DNS | May 12, 2008 4:24 PM

It's extra fun when you take a step back and look at the upshot of his "analysis": 50 percent of people thing Jeremiah Wright is an issue, which means that Jeremiah Wright determined 100 percent of the votes. Guh?

Posted by Jake H. | May 12, 2008 4:57 PM

They spent a lot of money on those polls. They *have* to get a story out of them. Thus, they are required to misinterpret them, because it's so hard to get a causal story out of a single poll will give you a clear causal story.

They know exactly how dumb their argument is, and so do many of the listeners, it's just a way to sell the poll results as interesting. As indeed they would have been if, say, Obama suffered a devastating slide and unexpected defeats post-Wright, and everybody cited Wright as the reason for that. The real reason the poll is so uninteresting is that Obama got about the votes you would have expected before Wright's latest appearance.

It kind of annoys me when random academics assume that everybody else is too dumb to understand the most basic issues in their discipline, when there are ample other explanations available.

Posted by mq | May 12, 2008 5:36 PM

Stephanopolous' formulation is deficient on another level.

If you tell me that:

Of those who said it was important, 70% went for Senator Clinton.
Of those who thought it was unimportant, 65% went for Barack Obama

I want to know how many "thought it was important" and how many didn't. As in numbers instead of percentages.

For all we know there were 10 voters who said it was important and of those 7 went for Clinton. On the other side of the question there may have been a million who said it was important, of which 650,000 went for Obama.

Without the number of voters in each category those percentages alone tell you less than nothing.

Posted by Curt M | May 12, 2008 5:36 PM

Come on. They paid money for that poll. They have to sell the results. Stephanopoulos isn't dumb, he knows everything Siegelman is saying but he's also smart enough to know what flies on TV.

Posted by mq | May 12, 2008 5:37 PM

whoops. Double post. Speaking of dumb. Except I don't think it's my fault -- when oh when are you going to fix this fucked-up server?

Posted by mq | May 12, 2008 5:39 PM

Stephanopolous has to trot out some stupid reason why his ABC debate sucked so bad. Its his way of defending himself from the inane questioning.

'see? these stupid issues mattered after all'

Posted by An Outhouse | May 12, 2008 6:04 PM

The post assumes that George is acting in good faith but stupid.

He is not stupid. He has acted in bad faith and the debate was the most telling yet...

He is a Clinton whore, co-conspirator racist McCarthyite trying to make their talking points part of the MSM dialouge.

Clintonism is realy Neoconism Lite

Posted by dana | May 12, 2008 8:20 PM

A Hilary supporter at work asked me before the IN/NC primary why the Wright story wasn't getting more play, and "How could he just sit there in the pew for years listening to that guy?" Then, the day after those primaries, he sheepishly informed me that he was switching candidates. Apparently, he forgot all about Wright.

Posted by peejay | May 12, 2008 8:49 PM

"Apparently, he forgot all about Wright."

Good for him!

And BTW, I don't think Stephalopololictiopolis' studity and dishonesty are mutually exclusive...

And for the love of God, Matthew, get Atlantic to fix the frickin' servers!

Posted by Rihilism | May 12, 2008 11:17 PM

There's a Freudian slip for ya. Studity? All right, I thought he was kinda cute once in a hobbity sorta way (betcha money he's got hairy feet). But Mr Steph...oh, fuck it, George was never very bright and became dishonest.

Posted by Rihilism | May 12, 2008 11:22 PM

"...Reverend Wright was an important issue because they knew it was an issue that reflected poorly on Obama."

Rev. Wright did not affect me negatively. The way the media and the opposition (from Clinton all the way to the wingnuts) handled it, and the way he was re-defined to the public so that Obama was obliged to distance himself, DID offend me.

Posted by Mellifluous | May 13, 2008 6:01 PM