Science Fiction Awards Watch

A New Hugo Award Podcast

Yes, we have been at it again. Kevin and Cheryl, ably assisted by John DeNardo of SF Signal, have recorded another podcast full of discussion of questions about the Hugo Awards. If you have difficulty listening to the podcast here you can access it via the Salon Futura iTunes feed, or download it directly using this link.

This time we have some extensive show notes for you. Those of you who like statistics, the numbers are all below the cut.

First of all, we promised you some pie charts indicating the nationality of the voters. Here they are:

2010

2010 Voter Origin Chart

2011

2011 Voter Origin Chart

Now here are some numbers. First of all, how many nominating ballots were received?

2011 Nominating Ballots

We see that while the overall increase in participation is a healthy 26%, that average masks significant differences between categories. The huge jump in interest for Related Work is perhaps due to the fact that podcasts are now eligible. Graphic Story is clearly picking up interest since its introduction. Interest in the Fan Artist category pretty much static.

Note that by no means all voters participate in all categories. It is possible (but unlikely) that the group of people who nominated in the Artist category are an entirely separate group of people to those who nominated in the Fanzine category. Because of this it is a mistake to think of “the Hugo voters” as a single, monolithic group with the same passions and biases.

It is easier to see which categories have a healthy level of interest if you express the numbers as a percentage of total ballots.

2011 Participation Rates

Novel is by far the most popular category, with Short Story and BDP Long putting in good showings. Fan Artist is clearly the least popular.

Our next chart shows the total number of nominations received in each category. Remember that each voter can nominate up to 5 works/people in each category. On average voters list between 2 and 3 works in each category.

2011 Nominations

A rather more interesting table is the number of unique nominations. That is, the total number of different works/people named in each category.

2011 Unique Nominations

Interestingly, Short Story has the largest number of unique nominations in every year. Editor: Long and Semiprozine have the fewest, though the charges that there are only a handful of works actually eligible for the Semiprozine category (made during the recent debate over the elimination of the category) are clearly false.

If we divide unique nominations by total nominations we get a metric that we might call the level of Uniqueness in the category. A category with a high degree of Uniqueness has a very large number of different works/people being nominated. As we shall see, this is not necessarily a good thing, because it makes it harder for consensus to form around any individual nominee.

2011 Uniqueness

Short Story and Graphic Story have the highest degree of Uniqueness. Consequently a candidate work in this category will probably need relatively few nominations to reach the ballot. The voters have great difficultly in agreeing which works are the best.

In contrast Semiprozine and BDP Long have the least level of Uniqueness. Here the voters have a great deal of agreement as to which are the best works.

Essentially what we are looking at here are differently shaped distributions of nominations. That for Short Story is very broad and flat, while that for BDP Long is very narrow and tall. You can see that from the minimum and maximum number of nominations achieved by the works that made the ballot.

2011 Minimum Nominations on Ballot

2011 Maximum Nominations on Ballot

If we divide the Minimum Nominations on Ballot by the number of Nominating Ballots we get the Threshold level which shows the level of support the final qualifying work received.

2011 Threshold

This is where the now notorious 5% rule comes in. A work much have at least 5% of the electorate supporting it to be allowed on the ballot. As we can see, Short Story is always very close to the mark. This year only 4 stories achieved that 5%.

It is worth pointing out that this rule has been in place since before Kevin started attending Worldcon (1984). There has not been a sudden conspiracy to discriminate against short story writers and unfairly deny them a place on the ballot.

Given the way fandom works, people are now expecting catastrophe. What happens if next year there are only 2 short stories on the ballot? Well to start with the WSFS Constitution mandates that at least 3 works must appear on the ballot, regardless of how much support they get. Also the data seems to suggest that the level of Uniqueness in Short Story is actually reducing. Had it been increasing with increasing numbers of voters there would have been cause to worry, but as it is decreasing it looks like this year was an aberration, not a sign of a developing trend.

Our final chart shows the Maximum Nominations divided by Nominating Ballots, which gives us a measure of the Peak Concentration in the category. This highlights those categories where it appears there is very little competition because certain works/people have the support of a very large portion of the electorate.

2011 Peak Concentration

In 2009 Wall-E had the highest number of nominations in the BDP Long category. It went on the receive more than twice the number of first-preference votes than the next most popular work, though it did not achieve an outright win.

Our thanks to the 2009, 2010 and 2011 Hugo Award Administrators for producing such detailed statistics. We hope that future Worldcons will follow suit.

10 Responses to “A New Hugo Award Podcast”

  1. […] Kevin and I have been at it again. There is a new Hugo Award podcast up at SF Awards Watch. In this one we explain why there are only 4 nominees in Short Story this […]

  2. on 17 May 2011 at 11:26 pmSF Signal: SF Tidbits for 5/18/11

    […] Next Web (Simon Owens) profiles Jay Lake. Science Fiction Awards Watch has a new Hugo Awards podcast.The Dragon Page interviews Bradley Beaulieu (podcast).Suvudu (Matt Staggs) Take Five with Coleridge […]

  3. on 19 May 2011 at 2:29 pmArno

    Thanks for an enlightening podcast. Having nominated and voted on the Hugos almost every year since 1975, I am always eager to better understand the process.

    I understand that the Hugos are popularity contests and that I will never agree completely on the nominations and winners. “Best” is subjective to each voter.

    I cannot read everything every year– no one can– but I try to be as conscientious as possible when nominating by looking at as many reviews and best-of-the-year lists as possible to make sure I don’t overlook a significant piece of work. So I feel that I am usually very well versed in what is popular, whether I agree with it or not. Yet it seems that every year there are works that get nominated that have not appeared on any of the lists I look at. I can only conclude that there must be some influential zines or blogs that I’m missing that are promoting these works. It’s hard to imagine that enough nominators could spontaneously nominate some of these obscure works.

    I don’t want to imply that any changes need to be made to the nominating process. I understand that I will never agree with every nominated work. I guess my question is: are there special interest groups that are able to mobilize enough people to get these fringe works nominated? Or am I just seeing a conspiracy where none exists?

  4. on 19 May 2011 at 3:02 pmKevin

    I think “conspiracy” is much too strong a word here, unless you mean that anything you don’t like being nominated must be a conspiracy. With thousands of potential nominators, nobody could possibly try and keep abreast of everything. There are groups of fans who discuss what they like and maintain lists of what they suggest would make good nominees: For instance, both BASFA and NESFA maintain recommendation lists. People mention in all manner of online fora works they liked and that they think deserve a Hugo, and sometimes those suggestions strike a spark with other nominators. But conspiracy? As in one person getting several thousand dollars to buy people a bunch of Worldcon supporting memberships so that his/her sock puppets can nominate? I don’t buy it. Yes, it (probably) happened once twenty years ago. Generally, such a conspiracy is relatively easy for an administrator to spot by the pattern of ballots received. I’m not talking about a family of four whose ballots all look alike and are submitted within minutes of each other. (That certainly happens, but there’s nothing illegal or IMO even immoral about it.) Basically, a “conspiracy” to get someone a nomination takes more work and cost than most people are apt to invest. Furthermore, encouraging more and more members to nominate raises the bar that much higher, so the solution to any perceived conspiracy is to get more people to participate.

  5. on 19 May 2011 at 9:20 pmArno

    Yes, conspiracy is too harsh a word–what I really meant was organized campaign. I’m not even close to suggesting that money is changing hands or anything illegal is going on. Let me reiterate that it’s not about agreeing or disagreeing with anything that gets nominated (in fact, there have been occasions when I really liked certain surprise works). What I’m trying to wrap my head around is how these obscure works find enough voters to be nominated.

    I read the Locus list, the NESFA list, the BASFA list, the preliminary Nebula ballot, various other SF award ballots, the Eisner Award list, this SF Awards Watch blog, Amazon.com best seller and critics’ lists, best-of-the-year anthology tables of contents, Goodreads.com recommendations, IMDb.com ratings, and various online SF and comics-related podcasts, blogs, and fanzines to survey the SF field so that I can seek out the more prominent candidates before I nominate. When works show up on the final Hugo ballot that are not on any of these lists, I begin to wonder how the reviewers and critics missed them and yet enough voters combine to get them on the ballot. Nominations don’t spring out of a vacuum. My first reaction is that someone must be actively organizing campaigns for these works–who are they and what outlets (fanzines, blogs, fan clubs, etc.) are they using for their campaigns?

  6. on 19 May 2011 at 10:13 pmKevin

    The reason “obscure” works can get nominated is that even now, with over 1,000 nominating ballots submitted, it can sometimes take very few nominations to make the ballot. Look at the “Minimum Nominations On Ballot” statistic above. When you can make the ballot with fewer than thirty nominations, just being mentioned on someone’s blog in a way that catches the fancy of a few Worldcon members is enough to get onto the ballot.

  7. […] Hugos are just around the corner, and here’s a special podcast from Cheryl Morgan and Kevin Standlee of the Science Fiction Awards Watch to answer all your questions about how the Hugos work. And, because we neglected to mention it […]

  8. on 22 May 2011 at 3:21 amCheryl

    Having just checked the Hugo ballot, I can’t see any work on there that I would describe as “obscure”. The are works I haven’t read, and some of the short fiction I don’t recall seeing anyone talk about. But all of the short fiction is from well-known, reputable magazines. I can see someone complaining that Fan Artist nominees are “obscure” if they don’t read fanzines, but the idea that any of the nominated works this year are so unusual as to raise eyebrows seems completely bizarre to me.

  9. […] Kevin and I explained on our podcast last week, the Campbell is “not a Hugo” because it doesn’t belong to WSFS. The […]

  10. […] See also: A New Hugo Award Podcast, with Cheryl Morgan, Kevin Standlee and John DeNardo. […]