In recent decades, opinion surveys have become the primary means through which journalists and scholars assess the preferences of the public – above all during election campaigns and when major policy questions are at issue. Supporters argue that polls are the only way for the opinions of all citizens to be measured. But a growing number of analysts are pointing to the ways in which public opinion poll results can be contingent and incomplete. And some theorists are raising even more fundamental questions about what polls really measure. The shortfalls as well as strengths of polls are important to consider, especially now that ever more of them are sponsored by campaigns and advocacy groups seeking to advance their own agendas.
Polls as Important Tools of Democracy
When polls ask Americans how they feel about surveys, many express suspicions about methods and results. But this runs counter to the long-time celebratory views of most opinion researchers. George Gallup, one of the earliest and most famous evangelists for opinion research based in sound sampling methods, argued that polls offered lawmakers a unique, unbiased window into the true preferences of their constituents. Without polls, he said, officials would be guided “only by letters to congressmen, the lobbying of pressure groups, and the reports of political henchmen.”
This view has long been echoed by leading political scientists. In 1942, V.O. Key declared that the “poll furnishes potentially a means for the deflation of the extreme claims of pressure groups and for the testing of their extravagant claims of public sentiment in support of their demands." And Sidney Verba echoed these sentiments in his 1996 presidential address to the American Political Science Association, where he argued that survey research has the potential to counter inequalities of political participation based on economic resources and further “the democratic ideal of equal voice.” Surveying the views of all Americans, researchers believe, can take account of the concerns of those who might otherwise be ignored in the public sphere.
Systematic Biases in Polls
But numerous and proliferating constraints in contemporary survey research present barriers to realizing such democratic ideals.
- The challenge of reaching a fair sample. Traditional survey methods based on calling phones at random are now faltering because of the spread of cell phones and growing distrust and refusals to cooperate with phone pollsters. To correct for such issues, many pollsters now use some combination of telephone, mail, and internet samples – and statistically adjust results to better approximate views of the entire public. Yet these new approaches require an ever-greater set of debatable assumptions, making it hard to interpret and compare polling reports.
- How questions are asked shifts results. Survey researchers have long known that slight modifications in the phrasing of questions or response categories can significantly affect poll results. Reputable pollsters devote considerable time and effort to testing, refining, and standardizing questions. But when polls for hire are done on behalf of corporations, advocacy groups, and partisan campaigns, it is easy to sway the results.
- The problem of omitted respondents. Surveys that gauge support for a policy or candidate often fail to report on respondents who say they are "unsure" or "don't know," systematically understating or overstating true levels of support. According to political scientist Adam Berinsky, disadvantaged individuals are more likely to withhold answers on survey items and thus have their preferences downplayed.
- Do surveys really discover what people think? Paying attention to politics is time-intensive and costly, so some citizens know much more than others. With limited information, many survey respondents echo arguments they have heard in news coverage or rely on vague links to other issues and concepts with which they are more familiar. Surveys may not adequately reveal what fully informed opinion would be if citizens were better able to articulate their own interests and preferences.
Whose Opinions Matter – And What is Effective Public Opinion?
Even the very best designed surveys make assumptions about whose opinions count – assumptions that may or may not capture what is politically important. Most election-related polls, for instance, seek only to assess the preferences of registered voters likely to vote in a given election – or sometimes focus even more narrowly on “undecided” likely voters.
On public policy matters, the concerns of the public at large surely differ from those of the most concerned activists – whose views may matter more in the policy process. Sociologist Herbert Blumer once famously critiqued polls for treating society as if it were "only an aggregation of disparate individuals,” which ignores how opinions actually function in a society where some citizens carry more influence with policymakers than others. A recent study by political scientist Susan Herbst found, for example, state government officials tend to discount polls and pay more attention to media reports and the policy demands of interest groups. And political scientist Martin Gilens has found that politicians defer much more to the opinions of the most affluent Americans than to the views of the middle class and the poor.
Some scholars argue that the organized "public" dimension of politics deserves to be weighted more heavily. They criticize polling for over-stressing private fleeting individual impressions, while minimizing the voices of organized groups agitating for change. In America’s past, groups and organized social movements were seen as the bedrock of participatory democracy. An overreliance on incessant, momentary polls as the sole measure of public opinion may fail to appreciate the dynamic nature of citizen attitudes, variations in the intensity of opinion, and the capacity of organizations to express and advance particular public views more effectively than others.
In sum, there are many issues to ponder about the accuracy and ubiquity of polls. Although surveys remain an important tool for understanding certain aspects of mass preferences in American public life, it has become more important than ever to be cognizant of their many limitations and pitfalls.