The following study of ballot formats and voting errors was posted on
another listserve, and anyone who still cares about the matter, should find
it of interest.

Joel Redfield

----------------------------------------------------------------------------

The Butterfly Ballot Causes Confusion and Systematic Errors
in Voting Behavior
Robert C Sinclair

University of Alberta
Melvin M Mark
The Pennsylvania State University
Sean E Moore, Carrie A Lavis, Alexander S Soldat
University of Alberta

Two experiments investigated confusion and bias caused by
the butterfly ballot format used in Palm Beach County in the 2000 US
presidential election. In Study 1, Canadian students voted for Prime
Minister of Canada on a single-column or butterfly ballot. They rated the
butterfly ballot as significantly more confusing than the single-column format;
however, they made no voting errors. Study 2 replicated the confusion
effect with a nonstudent sample. Of greater importance, participants made
errors only on the butterfly ballot. The butterfly ballot causes confusion
and systematic errors in voting.

         The issue of systematic bias as a result of ballot
format has become the focus of much controversy surrounding the outcome
of the recent presidential election in the United States. Specifically,
people have argued that the format of the ballot in Palm Beach County
led to confusion and caused people who intended to vote for Al Gore to
mistakenly cast votes for Pat Buchanan or punch two holes resulting in a voided
ballot. We conducted two experimental studies to address this issue.
         On Wednesday, November 8, 2000 (the day after the
presidential election), we had Canadian college students vote for Prime
Minister of Canada using a single-column ballot format or a dual-column,
butterfly format (analogous to the Palm Beach County-style ballot). We
expected that students would rate the butterfly style as more confusing
than the single-column format. However, it was unclear whether
students, who are familiar with confusing optical scoring forms, would make
errors on the ballot.
Participants
         Participants were 324 introductory psychology
students from two classes at University of Alberta. All were volunteers who
participated in order to partially fulfill a course requirement.
Procedure
Ballot Construction.
The ballots contained the names of the
leaders of 10 Canadian political parties and space for a
write in candidate. One ballot used a single-column format. The
second was designed to emulate the dual-column, butterfly format used in Palm
Beach County (at the time this study was conducted, to the investigators
knowledge the actual ballot was not available on the web or in print
media, and the ballot was constructed after seeing it displayed for a brief
period on CNN). The butterfly ballot was designed so that the leaders
of the 2 predominant parties appeared in the first and second
positions in the first column. Specifically, Stockwell Day, leader of the Canadian
Alliance Party, was in the first position on the ballot, corresponding to
George Bush on the Palm Beach County ballot, and Jean Chretien, leader of
the Liberal Party of Canada, was in the second position, corresponding
to Al Gore. The leader of a third party, expected to receive few votes, was
the first name to appear in the second column. Specifically, Joe Clark,
leader of the Progressive Conservative Party of Canada, was in the
position on the ballot corresponding to Pat Buchanan on the Palm Beach County
ballot. The remaining candidates were also from parties expected to
receive few votes.
         The data were collected in large classrooms at
University of Alberta at the beginning of class. Participants were told
that we were interested in political issues. They were told that we were
holding a mock election for Prime Minister of Canada (there was,
conveniently, a federal election in Canada within 2 weeks following our data
collection).
Participants were randomly assigned to receive one of the
two ballot formats. They were asked to vote for Prime Minister by
darkening the circle beside the preferred candidate s name. Upon turning to the
next page, they were asked to report the degree to which the ballot was
confusing (using 2 items on 7-point scales, with high scores indicating greater
confusion), and to write out who they had intended to vote for on the
previous page. Finally, participants were debriefed.

         The mean of the two confusion items formed an index
of confusion (Cronbach s alpha = .96). Participants in the butterfly
format condition (M = 3.69) rated the ballot as significantly more confusing
than did participants in the single-column format condition (M =
2.14), t(322) = 8.23, p < .0001. No students made errors on the ballots.

         The results of Study 1 demonstrate that the Palm
Beach County-style ballot is perceived as significantly more
confusing that the single-column ballot. Generally, of course, greater
confusion is likely to lead to a higher error rate. This was not the case in Study
1; however, we were not surprised by the lack of errors given that our
sample involved college students who are quite skilled at completing
confusing optical scoring sheets. Thus, we decided to move our data collection
off campus and conduct a second study.
Participants
         Participants were 116 people recruited in a large
shopping mall in Edmonton, Alberta, Canada. There were 51 males and 62
females (three respondents failed to report their gender). The mean age was
51.10 years (sd = 19.19, range = 19-86; six respondents failed to report
their age).

Procedure
         Ballot Construction. The ballots were designed in the same manner
as described above. However, by Thursday, November 9, 2000,
we were able to view, in detail, the Palm Beach County ballot. Thus, the
butterfly ballot was in exactly the same format as that used in Palm Beach
County (with the exception that we did not use punch holes).
         Participants were approached individually at a mock
polling station set up in a busy shopping mall and asked to
participate in a Political Issues survey being conducted at the University of
Alberta. They were told that we were holding a mock election for Prime
Minister of Canada. Participants were randomly assigned to one of the
two ballot formats and were directed to one of two polling booths. They
voted for Prime Minister (as above). After turning the page in their
ballot, they reported the degree to which the ballot was confusing
(again, on 2 items on 7-point scales, with high scores indicating greater
confusion), wrote out who they had intended to vote for on the previous page,
reported their gender, age, and ethnic background, and placed their ballots
in a ballot box. Finally, participants were given a written debriefing
page describing the basis for the study and were offered a piece of candy.
Confusion Ratings
         The mean of the two confusion measures served as an
index of confusion (Cronbach s alpha = .81). The butterfly ballot was
rated as significantly more confusing (M = 3.44) than the
single-column ballot (M = 2.28), t(110) = 3.17, p < .003.

Errors
         We computed errors as a function of ballot type.  There were 4
errors, all of which occurred in the butterfly format, likelihood ratio
X2(1) = 5.27, p < .03. Interestingly, 3 of the 4 errors occurred for the
candidate who was in the same position on the butterfly ballot as was Al
Gore on the Palm Beach County ballot. This candidate's votes were
unintentionally given to the candidate who was in the same
position as Pat Buchanan on our butterfly ballot. Thus, the results suggest
that the butterfly ballot, as used in Palm Beach County, does result
in systematic errors.
         Both Studies 1 and 2 demonstrate that the butterfly
ballot used in Palm Beach County Florida is significantly more confusing
than a single-column ballot. Moreover, Study 2 demonstrates that
the Palm Beach County-style ballot systematically causes errors in voting
behavior. These findings call into question the validity of the results from
Palm Beach County in the 2000 US presidential election. With the ballot
style in use there, vote counts will systematically vary from the
intention of the electorate. In addition, the current findings may
underestimate the magnitude of bias. With a punch hole voting system,
imperfect alignment of a ballot in the voting machine might increase the likelihood
of errors with a butterfly ballot, given the proximity of punch holes
corresponding to the two columns. These findings are also likely to underestimate
the bias because the candidate in the first position on our butterfly
ballot (analogous to Bush) received 49.1% of the vote in Study 2
and no errors occurred in this position (the candidate in the second
position  corresponding to Gore  received 21.4% of the vote
and the remaining 8 candidates shared 29.5%).
It is not clear whether a biasing ballot format does or
should have legal standing in adjudicating disputes after an election. On the
other hand, given the centrality of elections to the democratic process,
it seems remarkable that biasing formats continue to be used. Low
cost application of social science theory and methods could help avoid such
controversies in the future.

Acknowledgements
         This research was supported by a Social Sciences and Humanities
Research Council of Canada operating grant to Robert C Sinclair. We thank
Sheree Kwong See for her assistance and Kelly Sinclair for comments on a
previous draft of this manuscript. Finally, we thank the administrators of
Bonnie Doon Shopping Centre for providing us with the space needed to
conduct Study 2.
 
 

***********************************************************************
Robert C Sinclair                       office: (780)
492-3822
Associate Professor                     home: (780) 436-0473
Department of Psychology                fax: (780) 492-1768
P-343 Bio Sci                           messages: (780)
492-5215
University of Alberta
Edmonton, AB T6G 2E9

http://www.psych.ualberta.ca/people/sinclair.html
 
 
 

***************************************************************************
Please sign all postings.

Please post PERSONAL replies or requests directly to the
message sender rather than to the entire list.

User settings, including email addresses, may be changed at
http://www.npsych.com
The list's administrator is dwloring@neuro.mcg.edu

List information can be obtained by sending a message to:
     lyris@neurolist.com
Type "help" as the subject for general instructions
Type "review npsych" as the subject for a member list

You may unsubscribe by forwarding this message to the
following address:
unsubscribe-npsych@neurolist.com or through the web address
above.

THE OPINIONS AND IDEAS PRESENTED ON THIS SERVER ARE THOSE OF
THE AUTHOR OF
THE MESSAGE. THE MEDICAL COLLEGE OF GEORGIA, THE STATE OF
GEORGIA, AND THE
NEUROLOGY DEPARTMENT TAKE NO RESPONSIBILITY FOR ANY
MISINFORMATION PRESENTED
BY ANY MESSAGE ECHOED BY THIS SERVER.