If you’ve looked at the myanimelist Top Anime list recently, you’ll notice that it contains anything but an even distribution of franchises across time. As of this writing, 14 of the top 30 come from the past 5 years, and 9 of the top 60 were pieces of animation which aired last year. The 2010-2014 period saw a lot of anime being produced, but as impressive as that amount was, it was hardly 45% of the historical total. Obviously, these rankings have a decently strong recency bias, skewing somewhat heavily towards newer anime. That in itself it perfectly fine. These rankings aren’t meant to be a paragon of good taste, but just to accurately represent how the site’s large userbase as a whole feels about them. This recency bias is a product of several factors: the site’s young userbase, the increasing number of anime being produced (and becoming available via simulcast) in recent years, and (possibly) a change in the quality of the on-screen product. In other words, it’s a product of the value accurately representing what it’s built to measure – overall popular consensus.
The same cannot be said for the same rankings’ strong preference for sequels. Not counting the two which are full-on retellings (FMA: Brotherhood and HxH 2011), 14 of the top 30 are either a continuation or a spinoff of a pre-existing franchise. This sequel bias, a product of non-fans tending to drop a show while fans almost always continue, actively interferes with the pure score’s ability to represent what the average viewer likes, as there is not a single sequel that gets watched by an average sample of the anime-viewing population.
Granted, that no series ends up getting seen by a truly random sampling of people. Time is limited, so the average viewer relies on gauging PVs, promo art, and word of mouth against their own tastes to determine what they want to watch. Thus, 99% of a given show’s audience are people at least a little interested in what it has to offer. However, there’s a significant difference between the interest level of someone who’s willing to try a series for one episode and one who’s already been watching it for upwards of 4 hours. That’s a difference in interest floors any season 2 or 3 or 4 gets to reap the benefits of, as the sixes, sevens and eights fall away much faster than the nines and tens do.
As an literal indicator of what a series was scored out of 10 by its average viewer, the current MAL scoring system is fine. As a way of sussing out which series own the popular consensus, and which newcomers checking it for ideas should try, sequel bias creates a notably deficient system. Taking one example, it’s particularly unfair in scoring things that run all at once; the same show built for a 24 episode run will have less of a chance of getting into a toplist with a given cutoff than one consisting of 2 separate 12-episode cour. It’s certainly unfair if two shows of theoretically-equal quality and runtime have unequal odds of making the top whatever simply because one didn’t air continuously. It defies common sense to allow that sort of arbitrary trait to be the margin that separates two otherwise identical-quality shows.
The most obvious solution to the problem of sequels getting the fans-only treatment, merging all franchises into a single entry, is bad for a number of reasons. Myanimelist is, first and foremost, a database that allows people to list the anime they’ve watched, and continually appending seasonal entries it a headache for those tracking the episode count, to say nothing of how franchise movies would get mixed into that brew. Too, if you want to take to task the 50% of sequels which are below average, you still have to respect the other fifty. Occasionally it’s the second season that wins the peoples’ hearts, standing far above the rest, Clannad: After Story being perhaps the most oft-cited case. And there’s also a meaningful difference between a second season that at least keeps the balls in the air and one that drops them into a sea of fire; it’s not like fans rate sequels on fundamentally different criteria than they rate the first season, so the differences in one person’s given scores remains a meaningful value. I personally can name two franchises where my rating jumped 5 and dropped 6 points between seasons, for good reason, and appreciate being able to separate those two.
The good news is that there is a fairly simple way to accommodate great sequels while noting the ones putting for pars or bogeys, and the necessary stats for it are already more or less built in to myanimelist’s system as it stands. In addition to scoring all shows they view, the site allows users to designate 5 series in particular as “favorites”, effectively allowing them to differentiate between their 10/10 scores and their 10+/10 scores. This allows us to get a better look at how sequels perform among fans; in theory a sequel surpassing the original should get proportionally more favorites, whereas a
To check whether or not this was true, I measured the number of total members, number of favorites, and score for shows at positions 1-30, 301-330, and 601-630. I then split each group of 30 up into sequels and non-sequels and calculated the average percentage of total members which had favorited the show for each group. The results, shown in more detail here, strongly support the assumption that favorites are far more frequently given to a show’s first season, even when accounting for a sequel’s overall decrease in popularity:
Controlling for score and popularity, an average sequel series on myanimelist is 2 to 5 times less likely to be favorited than a non-sequel counterpart. This correlation doesn’t hold for every case; Clannad: After Story and K-On!! both see strong favorite percentages relative to the rest of their group. That’s expected, since some sequels will just be better. The difference is robust enough that it can be applied towards a new, better ranking system.
There are a couple of mathematical ways to apply this idea, none fundamentally better than another. I settled on on averaging in the favorites F for a series with average score S and total members M as 11/10 scores, like so:
X= (S*M+11*F) / (M+F)
Let’s take a look at how this would work on the current toplist. This is what the top 30 looked like when I pulled it a few days ago:
You’ll notice there’s a good number of sequels to both Gintama and Mushishi ranked slightly above the original. Not that both series aren’t great, but it’s dubious as to whether they’d fare any better than the originals against a neutral panel of judges. Here’s how the same 30 would look like after being re-sorted using the favorites as a corrective factor as described above:
Here, the different installments of the Mushishi and Gintama franchises end up positioned closer to each other (as one would expect of series that maintain consistent quality from season to season), and two new series (Tengen Toppa Gurren Lagann and Death Note) jump up into the top 20.
This method works just as well outside of the top 30. Here’s what the ordering of series series in the original 301-330 group looks like before (note that Stardust Crusaders s2 is an oddity because of its newly-aired status at the time of the sampling):
Utena, formerly stuck in the middle of the pack, rises all the way to the top, and the top 5 goes from only having one original to being 100% original. That seems to me like a better way to build a list of shows people should be watching.
There are two legitimate concerns about changing the scoring system that I can think of. The first is that it would require additional processing power, adding in a step of calculation every time the site recomputes scores for the 10,000 or so titles it has listed. The site always seems to have problems of one sort or another, and they recently made it more difficult for users to search their entire database primarily for bandwidth reasons.* Though the math still wouldn’t a huge drain on resources, it is at least a valid point of discussion.
The second is that it could change the way people use their limited favorites from a passive, personal activity to a combative one that takes into account what other users are doing. While not trivial, there’s reason to argue the worst of that worry won’t come to pass. First, the majority of people vote their own feelings, not actively trying to give a certain show a certain rank. Second, favorites are limited commodities (only 5 allowed per user) which would limit the effectiveness of their usage – one user could focus on a franchise and try to boost all of it, but they would face a cost, hurting their other favorites. Such actions would be unlikely to move the needle much unless people undertook them by the thousands.
Rules changes usually produce long-term, unforeseen consequences, and this particular change is certainly something that should be considered carefully before having any chance of being implemented. I argue that it is at least worth some consideration, as we should be very meticulous in preserving the accuracy of statistics people use to size up anime at a glance.
*A full database search can still be run, provided one enters in a pair of blank spaces into the search bar.