Advanced Search

As the conventions end and the campaign season moves to the main event, the voting public will be showered with polls on a daily basis. The cacophony of contrasting polling numbers, margins of error, and different slices and dices of the electorate can often leave the interested observer more confused than informed.

I therefore thought it would be helpful to convey a few simple rules that professional poll watchers rely on to make sense of the numbers, and some common errors that even media outlets that should know better consistently fall into.

Polls Are Plural, Not Singular

Never pay too much attention to any one poll—look at an aggregation of polls over time. Polls fluctuate up and down; that is their nature. Over time they may hit their mark, but any one poll may be high or low due, if nothing else, to the vagaries of chance.

On Aug. 5, for instance, a CBS News poll had Barack Obama up by six points; five days later a Rasmussen poll had him up by only two points, within the margin of error. Much was made of this shift, coming as it did at a time when national energy policy and offshore drilling were headlining the news. Many pundits declared that Obama's lead had evaporated, that the energy issue had made him (and all Democrats) vulnerable, even going so far as to declare the shift a turning point in the race.

Seasoned observers thought it unlikely that Obama's lead had really disappeared, especially in the middle of the summer when not that many voters are paying attention to the race. Sure enough, the following day a Gallup poll had Obama up by five points, and it was the short-lived John McCain boom that dissipated.

Polls Need Context

Never give undue credence to poll questions that are not asked consistently over time. The best way to understand poll numbers is in the context of a series, not as a one-off set of responses.

The fact that President Bush's approval rating fell to the mid-20's, for instance, is significant, since pollsters have been asking the exact same approval question for decades, and Bush's numbers were the lowest recorded in modern history; lower even than Nixon's during Watergate.

On the other hand, much was made of a July 16 New York Times poll that supposedly showed a large gap in racial perceptions of politics and the state of society. When asked "Are race relations in the United States generally good or generally bad?" white respondents came out 55 percent good and 34 percent bad, while black respondents answered 29 percent good and 59 percent bad. This seems dramatic, but how are we to make sense of these results? We do not know how respondents in previous years would have answered, and it is likely that whites evaluated race relations in comparison to how far the nation has come in overcoming racism, while blacks were more mindful of how far we still have to go to ensure all Americans an equal opportunity to succeed. Without context, it is difficult if not impossible to interpret the responses.

Polls Are a Snapshot of a Moving Target

Public opinion does move, and it moves fastest in close proximity to an election, especially when voters had not paid much attention beforehand. Thus, seasoned poll watchers place little value in polls taken right before an election, especially a primary.

In the five-day gap between the Iowa caucuses and the New Hampshire primary, many polls had Obama ahead, which made Clinton's victory all the more startling. In fact, many commentators spoke of Obama's impending victory, putting all their faith in the polls that showed him ahead.

But a good number of voters probably made up their minds in the day or two before the primary, and there is no way a poll can accurately capture the impact of last-minute events, like Clinton's showing emotion on television the day before the vote, or the counter-reaction to Obama's new-found popularity. Most people I talked to considered the election a toss-up the days before the vote and were not overly surprised by Clinton's win.

People Lie to Pollsters

Polls are statistically correct, within their margin of error, only to the degree that they accurately measure the opinions of a random sample of voters. The sample sizes are not the problem (due to the magic of statistics, a mere 1,000 voters in a random sample can accurately reflect the opinions of the 120,000,000 or so actual voters on Election Day), but everything else is.

First of all, pollsters have to guess who the "likely" voters are, and this is as much art as science. People who voted before are likely to vote again, but what about young voters who never had a chance to vote previously? Or what about the fact that turnout has been higher in the primaries than ever before, indicating that general election turnout will probably be up as well? It all comes down to a guess as to who will vote, and no statistical technique can tell you which formula is better than the others.

Also, a dirty secret of the profession is that people lie to pollsters. They usually lie about things that embarrass them, like whether they voted previously, for whom they voted, and whom they intend to vote for in the next election, if they think that one answer is more socially acceptable than another.

Polls, taken with a grain of salt, can help voters understand trends and movements in a fluctuating election. To keep yourself sane, though, make sure you look at many polls at a time and beware the single-shot poll questions that usually obscure rather than capture or clarify reality.

 

Epstein, director of the Center on Political Economy and Comparative Institutional Analysis, is a professor of political science.

Top   |    E-mail this story