One of the many controversial features of sites like myanimelist and aniDB that allow users to list their anime is their inclusion of toplists. What’s the proper way to weight scores? Should sequels (which have an intrinsic advantage in 10-point averages) be counted normally? Is there a point to having one at all when it invites as much vitriol as it sometimes does?

Though actual discussions over topics like these tend to descend into unglorified hoopla fairly quickly, these toplists and rankings can be very interesting subjects for study. Especially if you dig a layer below the top and start to look at what they really measure.

My view of toplists is more than a bit off from the ordinary one. Most people look at the rankings and assume that shows ranked highest are the best shows, objectively. However, a quick eye test of the mal top 10 puts some screws to this line of argument. Gintama is good, but it’s not 3-of-the-best-10-anime-ever-made good. Also, it’s worth noting that 5 of those 10 (not counting FMA Brotherhood) are sequels. I had a footnote explaining why that was sketchy before it grew to about 700 words and became its own article.

1. Gintama’

2. Fullmetal Alchemist: Brotherhood

3. Steins;Gate

4. Gintama”

5. Clannad: After Story

6. Gintama

7. Legend of the Galactic Heroes

8. The Disappearance of Haruhi Suzumiya

9. Code Geass R2

10. Wolf Children

You could’ve gotten the same conclusion just by looking at the rank of Neon Genesis Evangelion (#200 as of this writing). What’s even more interesting to point out is the score distributions of these 10 shows. Here’s the average percentage of scores given to the mal top 10.*

10-48.2%

9-25.7%

8-14.7%

7-6.4%

6-2.2%

5-1.3%

4-0.5%

3-0.2%

2-0.2%

1-0.8%

Avg. Number of Members: 96,123

Average Score: 8.92 +/- 0.46

Points I take from this data:

1. Even for the most unanimously-favored shows, 50% of people would disagree with the following statement; “_ was the best anime of all time”.

2. The 1/10 percentage jumps out of the otherwise exponential decay in score percentages, suggesting that these top shows suffer some backlash, and possibly deliberate downvoting, from being ranked so high.

I’m about to be making broad, sweeping statements here, so it behooves me to examine more than one case. While we’re at it, let’s go down the list a bit and take a look at how the figures for numbers 101-110*, see how they stack up against their top-10 counterparts:

10-23.0%

9-28.7%

8-27.1%

7-14.2%

6-4.3%

5-1.7%

4-0.5%

3-0.2%

2-0.1%

1-0.2%

Avg. Number of Members: 61,345

Average Score: 8.41 +/- 0.45

Points I take from this data:

1. The percentage of 10/10 scores has gone from one coin flip to two. That’s not a ginormous shift, and shows we’re still looking at shows beloved by a lot of people.

2. The difference in average scores between the top 10 and series 100 removed from it is 0.51 +/- 0.64 (errors add in quadrature), a statistically negligible amount.

3. Shows this far down have a much smaller “backlash” 1/10 percentage.

4. The difference in popularity between the two groups is fairly minor, less than a factor of 2.

This data, plus years of looking at all this hubbub in various capacities on various sites, has led me to a conclusion: toplist ranks do not measure show quality so much as they measure unanimity of opinion on show quality. This statement may be rather obvious, but has some profound implications. One of which is this: It’s boring to talk about Cowboy Bebop.

Right about now, you’re probably hunting down my address right now so you can stab me while I sleep, but let me finish. Cowboy Bebop is a great show. Absolutely. Unequivocally. Almost anyone would agree. But, and this is a big but, that “almost anyone would agree” is the sticking point. Most of the conversations on Cowboy Bebop I’ve had have been short affairs; a little bit of raving over the ending, a little bit laughing about Mushroom Samba, and an agreement that the show was just generally awesome. It’s fun, but there’s a layer of depth missing there. I don’t really get to know much about the other person, just that we agree on this point. Sometimes we get into the nitty-gritty of direction that’s done well within the show, but not many people actually start conversations like that, so that’s usually as far as it goes.

For contrast, let’s consider a show further down the list, Bakemonogatari at #116. If you’ve never heard of or seen any of it, here’s a good introduction. This show is built on equal parts heavy dialogue and pointed yet sparse visuals. It’s simultaneously a cute-girls harem anime and an arthouse experimental piece. These traits, and their specific application in the show, are not going to gel for some people, in more ways than one. They’re also going to sweet-spot the pressure points of a lot of other people. Because of the wider gulf of opinions the show prompts (while still being generally high quality), I’ve rarely had a conversation about Bakemonogatari last more than 3 minutes that didn’t leave an impact on me, let alone one in which I agreed fully with the other person.

What I’m ultimately trying to get across here is that these lower (but still fairly high) ranked shows, those that aren’t first-ballot all-timers but still clearly well made, are, as an anime fandom, more worth talking about. While it totally makes sense to use the more unanimous top picks as recommendations for people just getting into anime whose tastes you don’t know, if you’re talking to someone who’s been watching anime for years who hasn’t seen it, it’s not a bad idea to stop and think before you bust out FMA Brotherhood as your first recommendation. Anime fandom is a lot more spread out and idiosyncratic than it is unanimous and clustered, and it behooves us to spend time on the idiosyncratic good titles and find out why we disagree. I’ve learned a lot more about my own tastes and others’ through the latter type of conversation, and I suspect most people would. And given how insular some parts of the fandom have gotten, I think it’s a healthy type of dialogue to have.

*These numbers may change with time. When I pulled them on 04/16/2013, the ones listed were as follows:

1. Gintama’

2. Fullmetal Alchemist: Brotherhood

3. Steins;Gate

4. Gintama”

5. Clannad: After Story

6. Gintama

7. Legend of the Galactic Heroes

8. The Disappearance of Haruhi Suzumiya

9. Code Geass R2

10. Wolf Children

101. Sword of the Stranger

102. Major Season 3

103. Nausicaa

104. Hunter x Hunter Greed Island Final OVA

105. Hunter x Hunter OVA

106. Fullmetal Alchemist

107. Rurouni Kenshin

108. Beast Player Erin

109. One Outs

110. Skip Beat