NES logo

November 25, 1997
To: The ANES Research Community
From: Charles Franklin, Chair, 1998 Pilot Study Committee
Larry Bartels, Chair, ANES Board of Overseers
RE: Call for Recommendations for the 1998 Pilot Study

The ANES Pilot Studies, which have been part of ANES research and development work since 1979, have been instrumental in the creation of much of the current content of the American National Election Studies. Examples of standard ANES content originally developed through pilot work include virtually all of the congressional elections batteries, the candidate traits and emotions batteries, the improved measures of religious affiliation and belief, measures of political knowledge and awareness, and many more.

We are soliciting ideas for an ANES pilot study that will be in the field under conditions that are unprecedented in ANES history: the study will be conducted in the context of an election year (1998). Up to now, ANES research and developmental studies have faced some important limitations because they have been conducted in odd numbered years, completely outside the electoral context. Given ANES' mission to study electoral politics and the impact of campaigns, taking this special opportunity to focus developmental research on items most specifically relevant to elections and campaigns -- indeed, some that can only be tested in the context of an electoral campaign -- constitutes a crucial phase in ANES' work. We seek the input of members of the research community in planning this project.

Below are several areas in which the ANES Board of Overseers would especially like to see active development work. We seek recommendations that address any or all of these specific areas or that address areas beyond those listed below which would specifically exploit the campaign setting in order to develop better measures of the campaign and its effects.

While the 1998 Pilot will be conducted during a midterm congressional election, the pilot study is NOT intended to focus on congressional elections per se, but rather to develop items applicable to all electoral settings, be they presidential, congressional or gubernatorial. In particular, it is hoped that items developed in 1998 will provide the basis for improved measurement during the 2000 presidential election study.

The key areas which the Board has identified for development work include (in no particular order): 1) candidate advertising, citizen exposure, and its impact, 2) the reasoning process by which voters translate often scanty information into vote choices, 3) the mobilization (or demobilization) process by which campaigns (selectively) affect turnout, 4) the efforts and effects of groups not associated with campaign organizations, especially in the area of contact with citizens and mobilization efforts, 5) the impact of debates and other campaign events on information and preferences, 6) the framing of issue agendas, 7) the role of election campaigns in social capital processes and democratic legitimation, and 8) improving measures of voter turnout.

A great deal of recent work has brought renewed interest in the effect of advertising in elections. This has been simulated by both laboratory experiments and new data sources on the distribution and frequency of advertising. The great barrier for exploring these effects in surveys has been the problem of how to measure exposure. Simple recall of ads is notoriously unreliable, while aggregate measures of what ads are broadcast suffer from an inability to measure differential exposure across individual respondents. In the 1995 Pilot and the 1996 ANES, a variety of new measures of media exposure were developed, some of which were directly aimed at providing unobtrusive yet more reliable measures of exposure to TV advertising. The analysis of these data is still underway. New work which extends these efforts, attempts to measure voter awareness of specific advertising messages, and yet deals successfully with problems of endogeneity is highly desirable.

2. The reasoning process

A variety of recent work has stressed the limited information upon which many voters base their decisions, and the cognitive processes by which they reach decisions. That work has primarily been developed using secondary analysis. We seek recommendations for items that can sensitively address the issues of cognition, inference, heuristics and rules of thumb which may serve as guides to voters.

3.The mobilization process

As turnout has declined, campaigns and party organizations have attempted only very selective mobilization efforts. In recent years, efforts at selective demobilization have also been reported. One obvious strategy for campaigns, if preferences are hard to change, is selective mobilization and suppression of turnout as more effective tactics. This modern campaign strategy has received scant attention in survey work, yet it seems an increasingly important aspect of campaigns. It also connects with the effect of negative advertising, which some argue can demobilize independents. Survey measures which would allow us to address these issues are needed.

4. The efforts and effects of groups not associated with campaign organizations

The increasing role played by independent organizations in electoral politics reached a zenith in the 1996 elections. Independent advertising, mobilization efforts, and fund raising have become a much more important part of campaigns than they were even a few years ago. Such efforts are particularly difficult to measure in surveys because of both the selective targeting of such efforts and because of respondent difficulty in discriminating between ads placed by candidates and those placed by independent groups. We hope to develop instrumentation which might allow estimation of the scope and effect of these efforts by independent organizations.

5. The impact of debates and other campaign events on information and preferences

Debates are now a stable part of presidential elections and are very common in state-wide and even House contests. Debates are unusual because they provide a fixed moment at which citizens can gather large amounts of politically relevant and directly comparative information about candidates. While we know a good deal about presidential debate effects, we have little instrumentation designed to specifically address the impact of debates on knowledge, interpretation and inferences. In a broader sense, debates are one particular type of campaign event. Either by use of debates as the specific example, or more generally applied to other events during campaigns, we seek instrumentation which can capture the role such events play in forming voters' final choices.

6. The Framing of issue agendas

Many scholars have recently argued that campaigns are as much about which issues are important as they are about what positions candidates take. Past measurement of the issue agenda of campaigns has been largely limited to generic measures of the most important issue of the campaign and to issue items designed to capture broadly important and stable policy debates, but not so much the specific topics discussed in a particular campaign. We need new measures which allow us to examine both how issue agendas are constructed by citizens during the campaign and what citizens learn about candidates' specific policy proposals.

7. The role of election campaigns in social capital processes and democratic legitimation

Recently a great deal of attention has been focused on whether stocks of social capital in the United States have been depleted. Whether and how campaign activities and campaign discourse are affected by social capital is not well understood. The feedback from election campaigns to social capital and other political cultural orientations is also deserving of systematic investigation. Campaigns are more than just the choices that people make; potentially they can serve as democratic rituals that "bless" the winner, infuse a sense of hopefulness in the citizenry, restore or maintain institutional legitimacy, and increase social solidarity. Campaigns could also fail to fulfill these functions. The Board welcomes innovative proposals that would elucidate how election campaigns are intertwined with aspects of the broader civic culture.

8. Improving measures of voter turnout

Survey respondents virtually always over-report voting turnout. This problem is not unique to NES, but it has proven particularly recalcitrant. Past efforts have included manipulating the social desirability of voting, via new question wording and most recently an experiment in getting respondents to first recall the details of election day and the polling place, before asking turnout. None of these experiments has substantially affected reported turnout. Further, validation of voting records in local election offices is subject to substantial measurement error of its own. One part of the problem is clearly the correlation between voting and willingness to participate in surveys, a fact that explains at least part of the over-report. But this doesn't seem enough to account for all of the over-report. In addition, there is a consistent tendency for respondents to over-report support for incumbent congressional candidates. We have addressed this by increasing the rate of post-election interviewing in order to minimize the time between election day and the post-election interview. However, further work might also be done in this area. We would like to solicit ideas from the research community for another attempt at improving these items, taking advantage of the 1998 election setting.

Although the Board identified these eight areas as needing particular attention, recommendations on other topics are strongly encouraged. We can imagine the electoral context provoking proposals to retest, sharpen our understanding of, and perhaps revise longstanding items and batteries. We especially would welcome suggestions that exploit the unique campaign setting of the 1998 Pilot in order to develop better measures of the campaign and its effects.

Format for Suggestions:

Suggestions may be from individuals or teams of scholars. Recently these have come from everyone from graduate students to senior scholars. A memo on a single topic or a number of topics, or a separate memorandum for each topic would be equally welcome. We are committed to the cumulative and collective nature of high quality research in the broad social science community; just as many data collection efforts and research projects over the years have incorporated questions and strategies initiated by the NES, we are also very interested in recommendations based on the fruits of other projects.

Although individuals and teams of scholars obviously submit ideas that speak to their own research interests, the Pilot Studies, like all NES data, are public data, designated by the National Science Foundation as a national resource, and therefore suggestions will be evaluated on their potential contribution to the research community more broadly. NES data collection projects are not and cannot be designed as "omnibus" studies; no portion of the surveys "belong" to any specific individual or group.

Your submission may take either of two forms. One would be a memo of general reactions to any of these ideas or other suggestions for consideration during the course of Pilot development. But especially if you have specific ideas about changes that should be made, new material that should be developed, or experiments that should be run, we suggest you take the second option: a concrete and detailed research recommendation. Here is how to assure the most favorable consideration possible:

Research recommendations should:

  1. justify the proposed new line of inquiry in theoretical or conceptual terms;
  2. lay out the proposed new instrumentation as clearly as possible (in draft form; if the idea is pursued there will be time for more work);
  3. sketch the kinds of analysis that will demonstrate how the new questions accomplish your research objectives. No pilot study is complete without a clear strategy for testing the utility of the proposed questions or strategies. You should outline the list of measures (e.g. which demographic data or other previous NES questions) that must be on the survey in order to do the necessary tests.

Memoranda that miss these elements are less likely to be convincing. Recommendations that accomplish this mission tend to be in the 5-10 page range (exclusive of appendices and bibliography).

The ANES Website has many resources that can help you design your suggestions including, among other things, recent interview schedules and data, the list of the NES question inventory (see the Continuity Guide), information about the recent pilot studies and abstracts of all the pilot study reports (some with full text of the reports available) written on the basis of the data collected, and all the recent embedded experiments.

The Board of Overseers will consider all submissions on their scholarly merits for possible inclusion in some form in the 1998 Pilot Study. We may engage in further consultations with you about your suggestions. Based on the submitted memoranda, the Board also expects to invite some members of the ANES community to join the planning committee for the 1998 Pilot Study and participate in the planning and evaluation of the Pilot data, and the preparation of reports and recommendations to the Board.

Your research memorandum must arrive in Ann Arbor by Monday February 2, 1998. It should be mailed to:

The 1998 Pilot Study
National Election Studies
Institute for Social Research
P.O. Box 1248
Ann Arbor, MI 48106-1248

You can also reply by email to (please make the subject line: 1998 Pilot Study) or by fax to: (734) 764-3341.

In the meantime, if you have any questions or if you would like to chat about an aspect of a proposal you are working on, please feel free to contact us or any member of the NES Board of Overseers. We look forward to working with you.