Polling as Witchcraft

One thing that's interesting to me about the way our politics works is that pollsters have this almost witchdoctor-like role in political campaigns. They're extremely important, influential advisers and they're wielding all this survey data and entrails and so forth, but at the end of the day they really seem to just be winging it every bit as much as anyone else. Take this from yesterday's NYT article on disagreements between Mark Penn and Geoff Garin:

Inside the Clinton team, Mr. Penn advocated increasingly sharp attacks on Mr. Obama as Mrs. Clinton’s best option. Long before he joined the campaign, Mr. Garin argued that her route to success lay more in presenting her strengths than in assailing her opponent.

This is obviously a big question, and yet two well-regarded pollsters have no kind of consensus over it, and it's clear enough that you're not going to resolve the dispute by surveying more people or staring longer at existing surveys. But so what, exactly, is the special authority of the pollster supposed to be?

Comments

It has to do with the average politician's (and average journalist's) math illiteracy. Pollster's use complicated formulas with greek letters, subscripts, and who knows what other stuff. Who's going to compete with that?

Posted by Carl | April 9, 2008 11:56 AM

Well, in this case Mr Garin's special authority consists in not being Mr Penn.

Posted by Ovid | April 9, 2008 11:58 AM

In the type of disagreement you are talking about, if a pollster has a specific point of view on how a campaign is to be run, they can do polling that will justify that point of view. Question design, etc, can produce a desired result.

Only when the object of the polling is to get a purely objective, non self-supporting result can it actually be considered relevant.

And looking at the PA polls and the all over the map aspect of them, even then they can be screwed up.

Posted by john m | April 9, 2008 12:02 PM

As someone who's worked in polling on the Democratic side for many years, I can tell you that actual polls have a very low profit margin. Advice on the other hand costs nothing to produce and you can charge lots for it. What would you rather do?

You could, however, test these two competing strategies through polling and focus groups and get a reasonably good sense of which is more effective. The problem in this case is that Mark Penn is notorious for manipulating his methodology to get the results that support whatever fad he's pushing at the moment.

Posted by AM | April 9, 2008 12:12 PM

1) I've seen one report that suggests the reason the Hillary Campaign ran off the rails is that Mark Penn was wearing two hats -- so that he himself responsible for evaluating how well a job he was doing. I'm not kidding.

2) That is, as campaign pollster, he was responsible for collecting the polling data and judging how effective his efforts as chief strategist were.

3) And as chief strategist, he was the one who decided whether the campaign pollster was doing a good job.

4) Hee hee. And with this setup, Hillary thinks she's competent to MANAGE the Executive Branch as President?

There's probably five defense executives with hardons thinking about the scams they could run if she was elected.

Posted by Don Williams | April 9, 2008 12:18 PM

Well, this sort of "polling" really is quite a lot like witchcraft, or actually more like having soothsayers examine the entrails of sheep to decide on whether a battle should be fought.

A bigger problem is that these pollsters are usually closely hooked up via a web of direct or indirect business relationships with the media consultants, who early gigantic fortunes from all political campaigns. Therefore, their advice usually amounts to something like "give my friend all your money so he can keep a big chunk of it to buy a fancier private jet"...

Posted by RKU | April 9, 2008 12:19 PM

Sorry, AM. Did mean to duplicate your point -- you posted while I was composing mine.

Posted by Don Williams | April 9, 2008 12:22 PM

Grrrr. This Yglesias Mad Cow disease is catching. Above post should have read:

"Sorry AM. Did NOT mean to duplicate your point"

Posted by Don Williams | April 9, 2008 12:27 PM

Ahem! In defense of witchcraft, I'd like to say, as someone who has lived for years in various African villages, that a "witchdoctor" is often far more accurate than either a pollster or meteorologists, etc. The reason for this is that diviners know the people in their village, know their family histories, their family dynamics, their personal psychologies. Invoking insight through spiritual means comes from a particular cultural perspective, but it is grounded in real local wisdom, and that is why it is so often successful.

Posted by Philip Graham | April 9, 2008 12:37 PM

The problem starts when people start believing that polls can predict the future. They can't, of course: they can only tell you what people are thinking at the time the poll is taken. And then when you add onto that basic present-versus-future mistake the idea that polls can tell you what will happen in the future based on alternative hypotheticals, you are really going to be easily misled.

Posted by DTM | April 9, 2008 12:38 PM

Does this mean we get to throw Penn in the river and see if he floats?

Posted by LaFollette Progressive | April 9, 2008 1:20 PM

Your post doesn't come close to making this point. Boiled down to its essence, your argument: When and if two "experts" disagree it proves their field isn't very useful.

That's of course absurd. In this particular case, one or the other man may be better at his job. And debatable interpretations of data come up in all sorts of fields.

Posted by dan | April 9, 2008 1:34 PM

Your post doesn't come close to making this point. Boiled down to its essence, your argument: When and if two "experts" disagree it proves their field isn't very useful.

That's of course absurd. In this particular case, one or the other man may be better at his job. And debatable interpretations of data come up in all sorts of fields.

Posted by dan | April 9, 2008 1:39 PM

This argument is confusing to me. Am I misreading this, or is the expertise of pollsters supposed to be polling? If so, we’ve got two experts in polling disagreeing about campaign strategy. Do we hire political scientists to fix our cars or auto mechanics to analyze our political system? Methinks too many people are wearing too many hats. Am I missing something?

Posted by signSansSignified | April 9, 2008 4:23 PM

"If so, we’ve got two experts in polling disagreeing about campaign strategy. Do we hire political scientists to fix our cars or auto mechanics to analyze our political system? Methinks too many people are wearing too many hats. Am I missing something?

Posted by signSansSignified | April 9, 2008 4:23 PM"

Except the whole point of polling is it is supposed to help direct the candidate towards a better strategy. The pollster is supposed to be knowledgeable enough that he or she knows what the poll results mean for a candidate. They aren't just a bunch of random numbers.

Posted by Reality Man | April 10, 2008 2:15 AM

The way I understand it, there are basically two types of polls: The accurate polls which are delivered privately to the customer, and only revealed by the customer if they chose, and the public polls, which are more exercises in public relations than polling. In some cases the pollster will run multiple wordings for the latter, and release the version which gets the best numbers for the customer.

Keep in mind that if you're looking at a poll, and you didn't pay for it, it's probably the latter sort.

Posted by Brett Bellmore | April 10, 2008 7:18 AM