What You Need to Know About Iowa Polls

Key lines from Jay Cost at RealClearPolitics with emphasis added:

I think the problem is that we do not have a reliable metric to measure the state of the race. Polls are of limited utility for gauging Iowa Democrats. This is a subject I discussed earlier in the year. There are two problems. The first is devising a sample of voters. Turnout in the Iowa caucuses is difficult to measure because it takes a good degree of devotion to participate. [...] A poll of Iowa Democratic caucus goers does not really mimic the process in which they participate. [...] Democrats begin by standing in an area designated for their first choice candidate. Then, for thirty minutes, they either persuade or are persuaded by others to switch their choices. At the end of the half hour, electioneering is halted and caucus officials count the number of supporters that each candidate has. Candidates who have less than 15% or 25% are deemed not to be viable. And so, another thirty minutes for electioneering is once again granted. The supporters of nonviable candidates must find new candidates to support, team up with supporters of other nonviable candidates to make their candidate viable, or abstain.

In short, the polls tell us that Hillary Clinton, John Edwards, and Barack Obama all have a large enough group of supporters in Iowa that they could win. They really don't tell us anything else, since the polling firms have no real ability to model what's going to take place.

Comments

Well, I think they can make some predictions based on past participation and strength of support.

Posted by bob | November 13, 2007 9:15 PM

Wouldn't a poll that measured caucus preference accurately simulate this process? Presumably, you would want to know the second and third and no-way-in-hell choices of Dodd voters, for example.

Posted by Klug | November 13, 2007 9:16 PM

I think some pollsters, maybe Zogby, put out some polls that try to do that, by asking for second choices and then adding those in for candidates polling below 15%. It only works somewhat as a simulator, however, as you don't really know which candidates will be viable and where.

Part of the problem is that there are some weird formulas based on geographical distribution that make it very advantageous to hit the viability threshold in very small town and rural caucuses.

Posted by bob | November 13, 2007 9:46 PM

What are these formulas, Bob?

Posted by Anonymous | November 13, 2007 10:09 PM

"They really don't tell us anything else, since the polling firms have no real ability to model what's going to take place."

Well, no.

While it is true that badly conducted Iowa caucus polls are far less useful than badly conducted primary polls.

But you can do decent Iowa caucus polling.

The DMR register poll the week before the caucuses is amazingly reliable.

And I'd guess a well conducted poll taken right now would be a good snapshot of where likely caucus-goers stand.

Just because it's hard doesn't mean it's impossible.

Posted by Petey | November 13, 2007 10:11 PM

Amazingly reliable? You mean the one that was off by 12 points for John Kerry, 9 points for John Edwards, 2 points for Howard Dean, and 7 points for Dick Gephardt?

http://www.realclearpolitics.com/Presidential_04/iowa_polls.html

And when you say amazingly reliable, do you have some sort of pattern of examples you are going on, or just that one?

Posted by Anonymous | November 13, 2007 10:46 PM

"Amazingly reliable? You mean the one that was off by 12 points for John Kerry, 9 points for John Edwards, 2 points for Howard Dean, and 7 points for Dick Gephardt?"

Yup. It got the order of the top four finishers correct. And it's done amazingly well in previous caucuses as well.

The numbers are always going to be way off because of viability issues and because the weird delegate allocation methods.

The entrance poll taken on caucus night was also far off in the final percentages, but also got the top four finishers correct.

The order of finish is all you can ask for in caucus polls. Getting the numbers right is impossible.

Posted by Petey | November 13, 2007 11:02 PM

The CNN Entrance Poll was much, much, closer, with differences that can reasonably be attributed to the caucus process itself.

http://www.cnn.com/ELECTION/2004/primaries/pages/epolls/IA/index.html

These results are similar to entrance polls from previous Democratic caucuses, with the eventual winners earning 3-5 more points than they poll at due to the vagaries of the viability rules.

Can you point me to a previous Selzer or Register poll of the Iowa Democratic caucuses that was "amazingly reliable?"

The fact is that Cost and Yglesias are correct - it is just damn difficult to poll the Iowa Democratic caucuses. If you did a sufficiently large sample with a sufficiently tight screen you could maybe model the results, but it would require a lot more work and a lot brighter people than are generally involved in modern polling.

Posted by Anonymous | November 13, 2007 11:33 PM

"The CNN Entrance Poll was much, much, closer, with differences that can reasonably be attributed to the caucus process itself."

The Fox Entrance Poll was halfway in-between the CNN entrance poll and the DMR poll at:

Kerry 29
Edwards 23
Dean 21
Geppy 16

So getting the final numbers is impossible even for entrance polls.

And yes, you are correct that the difference between the entrance polls and the final result can be chalked up to, in part, the caucus process rules.

-----

But more to your point:

Obviously an entrance poll should be much, much closer than a poll taken a week out of folks who necessarily have yet to show up or not show up at the caucus.

What Selzer is good at is getting is a pretty damn accurate snapshot a week out.

"Can you point me to a previous Selzer or Register poll of the Iowa Democratic caucuses that was "amazingly reliable?"

Unfortunately, I don't have a link for you. However, someone did post a comparison like this on the web recently.

I will note that Selzer is the gold standard for good reason. Well prior to the '04 results, everyone was cognizant that Selzer was the one to watch.

My pre-2004 caucus thoughts are immortalized here:

Post Des Moines Register Poll: 1st and 4th are taken. Congratulations to Kerry, and a gold watch for Geppy.

I wouldn't have been so definitive off of any other poll.

"The fact is that Cost and Yglesias are correct - it is just damn difficult to poll the Iowa Democratic caucuses."

Petey is in agreement with Cost and Yglesias that far. It is damn difficult. My only point is that difficult ≠ impossible.

Now, a further complication is that in a race like this, a poll from God himself two weeks before the caucuses wouldn't be of much predictive value since late deciders are going to make the difference here.

But difficult to poll or not, a good polling outfit can still get a reasonable snapshot of where likely caucus-goers are standing at the moment.

Posted by Petey | November 13, 2007 11:57 PM

What Fox News Entrance Poll? We are on the internet, provide links for your claims. Here is FNC claiming the exact same numbers as CNN:

http://www.foxnews.com/story/0,2933,108927,00.html

----

I agree that the Register Poll is highly respected. As far as polling goes, it is pretty good. But it was outside the margin of error for 3 of the 4 top candidates in 2004, which is not accurate by any stretch of the imagination. It had all four of the candidates within the margin of error of each other, just like almost every other poll from the time. It just so happened to either be slightly more accurate or to luck out and get the order correct.

In the 2004 general election, the poll had Iowa going for Kerry within the margin of error. It was just wrong there. That's the thing about margin of error - it really means something.

And I will ask again - can you point to any previous Democratic caucuses where the DMR/Selzer poll was "amazingly reliable?" I'm glad that you consider yourself a source worth citing, but I am looking for something slightly less biased. :-)

---

You're moving the goalposts, too. I don't disagree that a good poll can get a reasonable idea where likely caucus goers stand at the moment. While it is hard to get an appropriate sample, that is only one of many distortions introduced into trying to predict the winner of the caucus from polling. And that's what we're talking about.

Unless you can account for the other factors that we both agree exist - the viability requirements, and the extremely late decisions by a large number of caucus goers - you cannot accurately poll Iowa. And history bears this out.

Posted by Anonymous | November 14, 2007 10:55 AM