cross-posted at MyDD and Daily Kos

When I talk to friends or family from other parts of the country, they always want to know how I think the candidates are doing in Iowa.

This diary is about why that’s a tough question to answer.

First, I’ll discuss why opinion polls can’t necessarily tell us who would win the caucuses if they were held tonight.

Then I’ll explain why it can even be difficult for active volunteers to gauge who is ahead in their own neighborhoods.

Finally, I will go over the unscientific methods we foot-soldiers in Iowa use to figure out where our candidates stand.

In case you missed the earlier installments in the series:

How the Iowa caucuses work, part 1 (basic elements of the caucus system)

How the Iowa caucuses work, part 2 (corrects an error in part 1 and discusses who is over-represented and who is under-represented when delegates are counted)

How the Iowa caucuses work, part 3 (why it’s hard to turn out caucus-goers)

How the Iowa caucuses work, part 4 (more about why caucus turnout is low)

How the Iowa caucuses work, part 5 (on second choices and caucus math)

How the Iowa caucuses work, part 6 (on how precinct captains help their candidates before caucus night)

If you like political geography, you may want to know Where the Iowa field offices are. I have continued to update this diary as candidates have added more field offices this month.

Now, back to the topic at hand.

With new Iowa polls being released every few days, it can’t be emphasized strongly enough that it is very difficult to identify likely caucus-goers. Mark Blumenthal, who has forgotten more about polling than I know, has described the Iowa caucuses as “the pollster’s nightmare”. He has explained in detail why the likely voter screens adopted by pollsters can greatly affect the findings. (Incidentally, Blumenthal worked on Paul Simon’s campaign in Iowa before the 1988 caucuses.) Chris Cillizza covers the same problem in this post.

If you click on only one link in this diary, make it this post by Blumenthal about his efforts to obtain more information on pollsters’ likely voter screens and the demographic makeup of their respondents. Blumenthal demonstrates that

the differences in the way pollsters measure “likely caucus goers” in Iowa are huge, not just in how narrowly they define the electorate but in the kinds of voters pollsters select as “likely caucus goers.”

The second-choice option presents another challenge to pollsters. People backing candidates who have less than 15 percent support in a precinct can either join a different candidate’s group or try to attract enough people to bring their candidate up to that viability threshold. As I’ve discussed before, the second choices of caucus-goers can change the delegate counts in unexpected ways.

Many pollsters now ask respondents in Iowa about their second choices. But most caucus-goers won’t need to go to a second choice. Recognizing this fact, some pollsters only report the second choices of respondents who name Chris Dodd, Joe Biden or Bill Richardson as their first choice. That seems logical, because those candidates are polling below 15 percent statewide.

Trouble is, there will be hundreds of precincts in which one or more of the “second-tier” candidates are viable, and there will be hundreds of precincts in which one of the “top-tier” candidates is not viable. As Roger Simon recently noted, even the 2004 winner, John Kerry, was not viable in 222 of the nearly 1,800 precincts in Iowa. (Some of those were rural precincts that assigned only one delegate, so that only the candidate with a majority of votes in the precinct was “viable.”)

Perhaps the biggest reason it is hard to poll Iowa is that not everyone’s vote will carry equal weight on caucus night.

Remember, raw votes do not determine the winner. The winner will be the candidate who gets a plurality of the 2,500 Democratic state delegates.  

“Geraldine” put up a great post at Iowa Progress earlier this year on The Inequalities of the Iowa Caucus. The post demonstrates that in 2004, it took more voters to assign a state delegate in some counties than in others. I discussed this finding in an earlier diary, but I want to highlight this paragraph from Geraldine’s post again:

When one looks at the number of caucusgoers that it takes to elect a delegate in each county, there is a clear pattern. Caucusgoers in small, rural, Republican counties wield disproportionate power compared to those in more urban Democratic counties. In fact, the people who are most disadvantaged by this are students. Of the four counties where it required the most caucusgoers to elect a delegate [in 2004], three of them had significant student populations: Johnson, Poweshiek and Story. It is a system that favors the old over the young, the rural over the urban, Western Iowa over Eastern Iowa. It is a method that of selecting a candidate that has a clear bias and to be successful, presidential candidates have to spend a disproportionate amount of energy on less populous rural areas.

The most-populous Iowa counties may well have more than 50 caucus-goers show up on January 3 for each state delegate allocated to the county. In many smaller counties, perhaps fewer than 30 caucus-goers will turn out that night per state delegate allocated to the county.    

To complicate matters further, even within the same county, not everyone’s vote counts the same. A couple of months ago an acquaintance and I swapped caucus stories over lunch. He supported Dennis Kucinich and still regrets that his group fell just short of viability in his precinct. They needed two more people, and he couldn’t convince even his wife to come over to the Kucinich corner.

Wow, you must have had a lot of Kucinich supporters in your neighborhood, I said. (There were a handful at my caucus, but they would have needed two dozen more to be viable.) Well, he said, there were four of us, and we needed six to be viable.

Hang on–you only needed six people to reach the 15 percent threshold? That’s right. That means no more than 40 people showed up for his precinct caucus.

He isn’t from some remote rural area. He lives in Des Moines, the biggest city in Iowa’s biggest county (Polk). In my own Polk County precinct, we needed 27 supporters to reach the 15 percent threshold in 2004. There were precincts in Des Moines where candidates needed more than 40 supporters just to be viable. I couldn’t believe there was a precinct in this town where only 40 people turned out to caucus.

Since then I have contacted this acquaintance several times to try to bring him over to Edwards. He’s still undecided, considering Edwards, Obama and Richardson. I figure, if I can get him and his wife to caucus for Edwards, that will be worth as much as winning over eight or nine people in my precinct (assuming the disparity in our neighborhoods’ turnout is comparable this year).

Iowa City-based blogger John Deeth mentioned this aspect of the caucus system in a recent post at Iowa Independent:

For example in 2004, in Iowa City Precinct 18, a hotbed of activism full of liberal professors and students, 534 caucus-goers recreated the Black Hole Of Calcutta in the Longfellow Elementary School gym.  In North Liberty Precinct 1, full of trailer courts, newly developed housing and independents who marked their fall ballots for the Democratic ticket, only 171 people showed up.  But based on their general election voting in 2000 and 2002, North Liberty 1 and Iowa City 18 each elected the same total of 10 [Johnson County] delegates.

To put it another way, each person who showed up to caucus in North Liberty Precinct 1 influenced the allocation of Johnson County delegates as much as three people who caucused in Iowa City Precinct 18.

There is simply no way a pollster can weight for these disparities. A survey may find 25 percent support for Clinton, Obama and Edwards, but without knowing how those supporters are distributed across the state, it is hard to predict who will walk away with the most state delegates.

Since polls do not tell the whole story in Iowa, campaign staff and volunteers rely on direct voter contacts to gauge how their candidates are doing. All year, spreadsheets are updated as voters considered likely to attend the caucuses are identified as supporters, non-supporters, undecided or unlikely to caucus. I talked about the ways precinct captains help identify voter preferences in part 6 of this series.

I have a “vote goal,” which is the number of supporters the campaign thinks will be needed to put Edwards in position to earn the most delegates at my precinct caucus. My spreadsheet tells me how many people in my precinct have signed supporter cards for Edwards or have said they will caucus for him. We also keep track of voters who say they are leaning to Edwards.

There are bound to be some Edwards supporters I don’t know about, but just going by my spreadsheet, I am not at all worried about crossing the 15 percent threshold in my precinct. Even if turnout is considerably higher than it was in 2004, I know we will have at least 15 percent of the attendees in the Edwards corner.

Nevertheless, my voter contacts cannot tell me who would win my precinct if the caucuses were held tonight. Will Edwards have enough supporters to earn two or three delegates out of the six assigned by my precinct? Without knowing the total turnout, I can only guess. If we have 50 Edwards supporters, that would be enough for two delegates if 150 people attend my caucus, but it would not be enough for two delegates if 250 people turn up.

It’s even harder to figure out how much support other candidates have in my neighborhood. Some people don’t hesitate to tell me whom they support, but others prefer not to say. I label those voters non-supporters, but that doesn’t help me figure out which candidate is Edwards’ main competition in the precinct.

Some registered Democrats in my neighborhood never call me back and are not home when I knock on their door. Are they supporting other candidates, or do they simply not plan to caucus on January 3?

How many other candidates will cross the 15 percent threshold in my precinct? I know Biden or Richardson supporters who say Edwards is their second choice. They will only end up in our corner if their preferred candidates are not viable.

No matter how hard I work my precinct ahead of time, I won’t know how many delegates we’ll get for Edwards until the evening of January 3.

The unknowns can be frustrating, so volunteers and staffers look for other ways to figure out the state of the race in Iowa. Here are a few of these unscientific methods:

1. Track preferences of people who caucused in 2004

Most of the people who turn out on January 3 will have attended their precinct caucus in 2004. When I talk with politically active Iowans about the campaign, I usually ask whom they supported in 2004 as well as whom they support now.

All year I’ve been thinking that the path for Edwards in Iowa is to hold on to at least half of the people who caucused for him last time while bringing over a quarter of the Kerry supporters, a third of the Howard Dean supporters, and half of the Dick Gephardt and Kucinich supporters. This is just something I came up with, not based on any internal campaign strategy I’m aware of. It’s not the only path to winning Iowa, but it seems reasonable.

With this in mind, I don’t get bent out of shape every time I encounter someone who caucused for Edwards before but now favors a different candidate. I’ve had enough conversations with former Edwards supporters to feel very confident that he is holding on to a majority of those who were in his corner last time. (My field organizer and other volunteers get the same impression from their voter contacts.)

By the same token, I would be alarmed if the vast majority of people who caucused for Kerry were backing a particular candidate other than Edwards. Having been a precinct captain for Kerry, I know a lot of former Kerry supporters, and my sense is that they are all over the map, with no candidate dominating and Edwards getting a decent share of this group.

2. Track which candidates are getting serious consideration from undecided voters

My biggest regret from the last campaign was that I didn’t grasp sooner that undecided voters had ruled out Dean and Gephardt. For months I was demoralized by polls showing Dean and Gephardt way ahead of the others in Iowa, but I should have paid attention to all the people telling me, “I don’t know, but not Dean.”

I strongly disagree with the conventional wisdom that says Dean and Gephardt destroyed each other with negative advertising during the final weeks of the campaign. I have been unable to find a link to this video, but I recall seeing a C-SPAN panel discussion after the 2004 primaries, which included Gephardt’s campaign manager, Steve Murphy. He talked about focus groups the campaign convened in Iowa in September 2003. They were trying to figure out what message would break Gephardt out of the low 20s, but they could not crack the code. In fact, they repeatedly found that no matter what message they tried, as focus group participants learned more about all the candidates, they gravitated toward Kerry and Edwards–just as tens of thousands of Iowans did a few months later.

I can’t remember the name of the former Kerry staffer who was on this panel. He joked that he wished Steve Murphy had picked up the phone to tell him about these focus groups–it would have saved him a lot of anxiety later that fall (when all the opinion polls showed Kerry way behind in Iowa and New Hampshire). If anyone remembers seeing this panel discussion or can dig up a link to it, please let me know in the comments.

In any event, I have been predicting all year that Hillary Clinton would finish no better than third in Iowa. The biggest reason is the large number of people who have told me that they don’t have a candidate yet, but they know they won’t caucus for Hillary.

Similarly, I have felt all year that there was room for Biden and Richardson to significantly improve their standing in Iowa, because of the number of undecided voters who tell me they really like one or both of those candidates. I never believe any candidate has hit a ceiling until I stop running into undecided voters who might break that way.

A cautionary note: I try to ask all kinds of politically active Iowans (not just Edwards supporters) what they hear from the undecided voters they know. That’s because working from just one campaign’s list of undecided voters will skew what you hear.

The Obama supporter icebergslim recently wrote a diary about volunteering in Iowa. She made phone calls from the Obama field office in Cedar Rapids, and here’s one thing she found:

I noticed that of the undecideds they told me that it was between Obama and Clinton.  I was surprised at that, but hey, who knows.  I did not take a call where an undecided was with Edwards.  And what does that mean?

When I call through my list of dozens of undecided voters, I almost never hear they are stuck between Clinton and Obama. That’s not surprising, because if anyone told me or an Edwards field organizer earlier in the year that they were only considering Clinton and Obama, we would not code that person as “undecided.” We would code that person as a non-supporter, and the name would probably be removed from our lists for canvassing and phone banking.

On my list, lots of people still can’t decide between Edwards and Richardson, Edwards and Obama, Edwards and Biden or (less frequently) Edwards and Clinton. Most of those people wouldn’t be on a list of undecided voters that Obama volunteers such as icebergslim would call. The Obama campaign would probably already have coded them as non-supporters, with the exception of those wavering between Edwards and Obama.

3. Compare the organizational strength of the campaigns

All of the campaigns are working hard to identify supporters and make sure they attend the caucuses, but some candidates have more boots on the ground than others. I discussed this in my diary on Where the Iowa field offices are. Obama and Clinton have the most field offices by far. Clinton, Edwards and Obama have the largest contingent of field organizers (exact numbers are not available, as campaigns are continually adding staff in Iowa).

Meanwhile, the Edwards campaign has said it has precinct captains in more than 85 percent of Iowa’s precincts, and it has volunteer steering committees in all 99 counties. The group blog Iowa Independent has twice put Edwards at the top of its “Democratic power rankings” because of the continuing superiority of his grassroots organization. They noted in November that “Edwards started about a year ago with the best organization in Iowa, and most of the foundation he built here is still in place.”

Another cautionary note: we say, “Organize, organize, organize, and then get hot at the end.” On caucus day 2004, Des Moines Register political columnist David Yepsen predicted Dean would win based on his superior organization. But getting hot at the end sealed the deal for Kerry and Edwards.

4. Notice who gets mentioned by people who don’t “seem” political

I don’t expect a huge turnout on January 3, but I have kept my ears open this year for signs that voters who did not caucus in 2004 are excited about one of our current candidates. It seems that Obama and Clinton in particular would benefit from a massive turnout of people who have never caucused before. I attended a house party for Obama in May. Most of the people there were political junkies, but one woman hadn’t even voted in 2004 or 2006. She drove 20 minutes from her home on a weeknight to learn more about Obama after seeing him on Oprah. Is she still with him, and will she find her way to her precinct caucus?

Nate Willems, who was a regional director for Dean’s campaign in Iowa, has been volunteering for Edwards this year and observed last month:

In making calls through a list of rural Democrats who are consistent primary voters, but who lack a history of attending a caucus, my anecdotal notes show that Clinton is significantly stronger than any other candidate.  Accordingly, it does seem that she would benefit from a larger turnout.  

Amongst rural Democrats with a record of attending their caucus, my notes show a very competitive race between Edwards and Clinton with Obama distinctly behind.

I have a neighbor who votes in general elections but doesn’t caucus or vote in primaries (I contacted her several times before the 2004 caucus and the 2006 gubernatorial primary). This year, I’ve heard her say that she likes Hillary but isn’t sure whether she will attend our precinct caucus. Can the Clinton campaign find women like her and get them out on January 3?

5. Count endorsements from newspapers and public figures

I don’t think most voters are taking their cue from newspaper endorsements, but candidates often put quotes from them in their television advertisements during the final weeks. My sense is that these endorsements are most helpful to second-tier candidates, as they give voters leaning in their direction a signal that these are serious contenders. So, for instance, if Biden racks up a lot more newspaper endorsements than Richardson or Dodd, that could help him with undecided voters who are not thrilled with the top-tier candidates.

As for prominent public figures, former governor Tom Vilsack and his wife Christie have been campaigning for Clinton, longtime Attorney General Tom Miller and Des Moines Mayor Frank Cownie have endorsed Obama, and Congressman Bruce Braley has endorsed Edwards. Senator Tom Harkin and Congressman Leonard Boswell are not expected to endorse, and I haven’t heard anything about Congressman Dave Loebsack.

John Deeth has been keeping track of state legislators’ endorsements at Iowa Independent, most recently in this post, which also shows which candidates Iowa Democratic legislators endorsed during the last presidential campaign. I think Clinton and Obama may have each gotten one or two endorsements since Deeth’s chart was posted, though.

Again, I wouldn’t make too much of these endorsements, but it is striking that Biden has so many more than Dodd or Richardson.

6. Counting yard signs and bumper stickers around town

If yard signs told the whole story, Ron Paul would be ahead of Mike Huckabee among Iowa Republicans. So clearly, this isn’t the most important indicator of a candidate’s strength. Campaign staff never want to get bogged down in “yard sign wars” and never seem to have as many signs as volunteers would like to put up in their neighborhoods.

At the same time, paying attention to people’s yard signs and bumper stickers can help precinct captains identify the preferences of neighbors who don’t like to answer the phone. Also, I have no doubt that a wave of yard signs appearing for a particular candidate has a psychological affect on Iowans who are undecided or supporting other candidates.

(Side note on bumper stickers: an Edwards supporter in my neighborhood told me a few weeks ago that she saw a car driving in Des Moines with an “Impeach Hillary” sticker. How deranged are these wingnuts?)

When I mention some of my unscientific observations on political blogs, I sometimes am mocked for citing anecdotal evidence. It’s possible that my impressions and what I hear about other volunteers are way off base.

But I will never forget the day I called my Kerry field organizer in mid-December 2003. I had started finding Edwards supporters all over my precinct, a big change from October, when I only identified two people backing Edwards. I told my field organizer that Edwards was becoming a problem and would probably be viable in my neighborhood. He said, “I know.” They were hearing the same thing from all their precinct captains in the Des Moines area. This was several weeks before Iowa polls started showing Edwards’ rapid rise and the Des Moines Register endorsed Edwards.

Sometimes it’s worth listening to the word on the street.

Thanks to those who read this long diary. I look forward to reading your comments. In upcoming diaries, I will cover the different ways to win your precinct, explain what precinct captains do on caucus night, and respond to some arguments raised by those who like the caucus system.