Last night, the Iowa Democratic and Republican parties held the first in the nation caucuses to select the next President of the United States. As a high profile election, with lots of public polling, Iowa serves as yet another tedious referendum on the accuracy of polling (see here – “Will Trump Save the Pollsters” – and here – “Iowa Is the Hardest State to Poll” for some of many examples).
As with any poll, the first question a consumer should ask is “what is this poll telling me about the story of the race?” This is far more important than the 10 individual estimates of candidate support. The meta narrative about the Republican race in Iowa has been that caucus voters overwhelmingly prefer candidates from outside of government, or have positioned themselves as opponents of government from within (Trump, Cruz, Carson, Fiorina) than candidates from within government and have been involved in high profile bipartisan deals on important issues like immigration (Rubio, Kasich, Christie, Jeb!).
As far as this narrative, the polls were dead on. The outsiders got 63.1%, right on with the final estimate. The insiders got 29.5%, just slightly higher than the estimate.
The polling looks “off” because it got two unexpected results from a straight forward analysis of the estimates of voter support of each candidate. Every poll had Trump “ahead” of Cruz and Rubio significantly behind them. Cruz won and Rubio came in a strong third (a position that launched the presidencies of Lamar! Alexander in 1996, Alan Keyes in 2000, and Ron Paul in 2012).
Were these really “mistakes”? Only if you assume that the result in a poll is a precise estimate, (which it is not) and there is no sampling error (which there is). In the last nine polls in Iowa, Trump had a statistically significant lead over Cruz in just two of them. (Trump’s margin is in blue, Cruz’s in red). For polls with overlapping confidence intervals, you can not say that Trump led Cruz:
As far as individual polls, the Des Moines Register poll has a reputation for being the most accurate. How did they do?
Cruz and Rubio overperformed their poll result. Trump and Christie underperformed (albeit at barely the level of statistical significance). The pollster boldly declared that “there’s no indication of a [Rubio] surge: His support declined during the four days of polling.” That did turn out to be wrong, but entrance polls show that 29% late deciders broke to Rubio compared to 14% for Trump.
A late poll, by a college in Massachusetts using automated phones to call just 298 likely Republicans (not marks of reliability), caught the Rubio surge but offered no clear guidance which of the top three candidates would win (and got Carson’s support way off).
Rubio ended up getting 78.2% of the “insider” vote, much higher than his share of that vote in polling. The “insider” vote didn’t grow, it just coalesced around Rubio. If that happens next week in New Hampshire, a state where the Governors Kasich, Jeb!, and Christie have spent considerably more time and resources, he could come close, if not win. The outsiders edge here is much narrower than in Iowa (albeit is growing).
Polls will be much more useful for consumers if both the media outlets and readers adjust expectations in what is observable and what is not. Getting precise estimates of 10 Republican candidates is not realistic and should not be either the goal or the measuring stick. Close poll results will not predict win, place, or show. But they will give context to what is going on in the race and which factors will be determinate in answering the big questions.
One last note, polling showed a close Democratic race, and the result was a virtual tie. That should not be conveniently forgotten.