Funny Numbers
Do Polls Lie About Race?

By KATE ZERNIKE
Published: October 11, 2008
THREE weeks to Election Day and polls project a victory, possibly a big one, for Barack Obama.

Yet everywhere, anxious Democrats wring their hands. They’ve seen this Lucy-and-the-football routine before, and they’re just waiting for their ball to be snatched away, the foiled Charlie Browns again. Remember how the exit polls in 2004 predicted President Kerry?

The anxiety is more acute this year, because Senator Obama is the first African-American major-party presidential nominee. And even pollsters say they can’t be sure how accurately polls capture people’s feelings about race, or how forthcoming Americans are in talking about a black candidate.

In recent days, nervous Obama supporters have traded worry about a survey — widely disputed by pollsters yet voraciously consumed by the politically obsessed — that concluded racial bias would cost Mr. Obama six percentage points in the final outcome. He is, of course, about six points ahead in current polls. See? He’s going to lose.

If he does, it wouldn’t be the first time that polls have overstated support for an African-American candidate. Since 1982, people have talked about the Bradley effect, where even last-minute polls predict a wide margin of victory, yet the black candidate goes on to lose, or win in a squeaker. (In the case that lent the phenomenon its name, Tom Bradley, the mayor of Los Angeles, lost his race for governor, the assumption being that voters lied to pollsters about their support for an African-American.)

But pollsters and political scientists say concern about a Bradley effect — some call it a Wilder effect or a Dinkins effect, and plenty call it a theory in search of data — is misplaced. It obscures what they argue is the more important point: there are plenty of ways that race complicates polling. Considered alone or in combination, these factors could produce an unforeseen Obama landslide with surprise victories in the South, a stunningly large Obama loss, or a recount-thin margin. In a year that has already turned expectations upside down, it is hard to completely reassure the fretters.

Among the non-Bradley factors at the intersection of race and polling is something called the reverse Bradley (perhaps more prevalent than the Bradley), in which polls understate support for a black candidate, particularly in regions where it is socially acceptable to express distrust of blacks. Then there are the voters not captured by polls. Research shows that those who refuse to participate in surveys tend to be less likely to vote for a black candidate. The race of the questioner, too, affects a poll — but no one is sure whether people give more or less accurate answers when they’re interviewed by someone of their own race.

“How much we are under-representing people who are intolerant and therefore unlikely to vote for Obama is an open question,” said Andrew Kohut, the president of Pew Research Center. “I suspect not a great deal, but maybe some. And ‘maybe some’ could be crucial in a tight election.”

In 1982, exit polls had Mayor Bradley so likely to win that newspaper headlines called him the victor. Yet he lost, narrowly. There emerged what seemed like a pattern: a number of polls found more support than there actually was for Harold Washington in the 1983 Chicago mayoral race; for David N. Dinkins in the 1989 New York mayoral race; and for L. Douglas Wilder in the 1989 Virginia governor’s race.

Were people so afraid to appear bigoted that they lied to pollsters, thinking it more socially acceptable to support a black candidate? Pollsters and political scientists have long questioned that assumption because they do not believe people have an incentive to deceive unless they are explicitly asked, “Do you support the white guy or the black guy?”

“We have no evidence that people lie to us,” said Joe Lenski, executive vice president of Edison Media Research, which conducts the exit polls the television networks use. He and others say that discrepancy in the polls has more to do with which people decline to participate, or say they are undecided.

Adam Berinsky, a political scientist at M.I.T. who has written about the “I don’t know” voters, points out that while polls overpredicted Mr. Dinkins’s support in 1989, they got it right in 1993, when he was running against the same opponent, Rudolph Giuliani. In 1989, Mr. Berinsky argues, people who feared being thought racist said “I don’t know.” By 1993, they could find things in Mr. Dinkins’s mayoral record to object to and so felt more free to express their opposition without fear of seeming racist.

Mr. Kohut conducted a study in 1997 looking at differences between people who readily agreed to be polled and those who agreed only after one or more callbacks. Reluctant participants were significantly more likely to have negative attitudes toward blacks — 15 percent said they had a “very favorable” attitude toward them, as opposed to 24 percent of the ready respondents. “The kinds of people suspicious of surveys are also more intolerant,” Mr. Kohut said.

Scott Keeter, Pew’s director of survey research, said pollsters had a harder time reaching voters with lower levels of education. Less-educated whites are the kind Mr. Obama has had trouble winning over. Conversely, young people are more likely to answer surveys, and they tend to favor Mr. Obama.

There may be several factors at work: Michael Traugott, a University of Michigan professor who studies polling, argues that the Bradley effect was misnamed from the start; the problem with the polls in the 1982 race was not that they failed to capture latent racism but that they failed to account for the absentee ballots, which ultimately handed the election to the white Republican, George Deukmejian.

Whatever its causes, the Bradley gap seems to be disappearing.

In a new study, Daniel J. Hopkins, a postdoctoral fellow at Harvard, considered 133 elections between 1989 and 2006 and found that blacks running for office before 1996 suffered a median Bradley effect of 3 percentage points. Blacks running after 1996, however, performed about 3 percentage points better than their polls predicted. Mr. Hopkins argues that the changes in the welfare laws in 1996 and the decline of violent crime took off the table issues that had aggravated racial animosity.

The Bradley effect in the 2006 vote was largely absent (and in some stances a reverse effect was seen by some pollsters). In Tennessee, Harold Ford Jr., a black congressman, lost by six points. His pollster, Pete Brodnitz, said the campaign had been watching for a Bradley effect and screened carefully to make sure its own polls looked only at the people most likely to vote. Internal polls were largely correct, but some public polls, relying on a more general population, were wildly off. Mr. Brodnitz blamed bad polling, not lying.

In this year’s Democratic primaries, University of Washington researchers found a Bradley effect in three states, but a reverse Bradley effect in 12 (in the other 17, polls were within a seven-point margin of error).

The results tended to correlate with the black population in a state: blacks made up 15 percent or more of the population in almost all the states where the polls showed less support for Mr. Obama than there actually was; in the three states where polls showed more support than there was, less than 10 percent of the population is black.

The differences are too great to be explained by just high black turnout, said Anthony Greenwald, one of the researchers. Nor were people necessarily lying. Instead, he sees a cultural dynamic at work: the states where polls underpredicted support for Mr. Obama were generally in the Southeast, where the culture has more stubbornly favored whites, so the “right” answer there was to choose the white candidate. In the three states where polls in the study overpredicted support for Mr. Obama — Rhode Island, California and New Hampshire — “the desirable thing is to appear unbiased and unprejudiced,” Mr. Greenwald said. (Many polling experts also believe that Mr. Obama was benefiting from an Iowa bounce in the late New Hampshire polls, as Senator Hillary Rodham Clinton had been ahead for months, and that therefore Mr. Obama’s loss there was not a true Bradley effect.)

The Bradley effect, Mr. Greenwald concluded, “has conceptually mutated.” “It’s not something that’s an absolute that we should generally expect, but something that will vary with the cultural context and the desirability of expressing pro-black attitudes.”

A further complication is the race of the person who asks the questions. Talking to a white interviewer, blacks or whites are more likely to say that they are supporting the white candidate; talking to a black interviewer, people are more likely to support the black candidate. This holds true whether the surveys are in person, or on the phone.

It could be that people worry about offending the interviewer by suggesting, “I wouldn’t vote for someone like you.” Or, researchers suggest, talking to a black polltaker who sounds energetic or professional might prime positive images of blacks, overwhelming any negative stereotypes.

The trouble is, “We don’t know that doing white-on-white interviews and black-on-black interviews would be more accurate,” said Jon Krosnick, a professor of psychology and political science at Stanford. “It is possible that right now the social norms within the African-American community are such that if you’re going to vote for McCain, it’s too embarrassing to admit, and if you’re not going to vote at all, it’s almost as embarrassing.”

The question of how race affects polling is of course different from the question of how it affects the vote. Many experts argue that race does not play a huge role in either this year, because the economy has emerged as such a dominant issue, and Mr. Obama is not primarily identified by his race.

But most of what they know, they know from polls. And even in the least complicated years, polling is a recipe with a good dash of “Who knows?”