1 vs 1 vote: calculate ratings (Flickchart.com)

Question

Instead of rating items with grades from 1 to 10, I would like to have 1 vs 1 "fights". Two items are displayed beside each other and you pick the one which you like more. Based on these "fight" results, an algorithm should calculate ratings for each item.

You can see this approach on Flickchart.com where movies are rated using this approach.

It looks like this:

As you can see, items are pushed upwards if they win a "fight". The ranking is always changing based on the "fight" results. But this can't be only based on the win quote (here 54%) since it's harder to win against "Titanic" than against "25th Hour" or so.

There are a few things which are quite unclear for me: - How are the ratings calculated? How do you decide which film is on the first place in the ranking? You have to consider how often an items wins and how good are the beaten items. - How to choose which items have a "fight"?

Of course, you can't tell me how Flickchart exactly does this all. But maybe you can tell me how it could be done. Thanks in advance!

Answer 1

This might not be exactly what flickchart is doing, but you could use a variant of the ELO algorithm used in chess (and other sports), since these are essentially fights/games that they win/lose.

Basically, all movies start off with 0 wins/losses and every time they get a win they get a certain amount of points. You usually have an average around 20 (but any number will do) and winning against a movie with the same rating as yourself will give exactly that 20. Winning against a bad movie will maybe give around 10 points, while winning against a better movie might give you 30 points. The other way around, losing to a good movie you only lose 10 points, but if you lose to a bad movie, you lose 30 points.

The specifics of the algorithm is in the wikipedia link.

Answer 2

How are the ratings calculated? How do you decide which film is on the first place in the ranking? You have to consider how often an items wins and how good are the beaten items.

What you want is a weighted rating, also called a Bayesian estimate.

I think IMDB's Top 250 movies is a better starting point to make a ranking website. Some movies have 300,000+ votes while others others have fewer than 50,000. IMDB uses a Bayesian estimate to rank movies against one another without unfairly weighting popular movies. The algorithm is given at the bottom of the page:

weighted rating (WR) = (v ÷ (v+m)) × R + (m ÷ (v+m)) × C where:

• R = average for the movie (mean) = (Rating)
• v = number of votes for the movie = (votes)
• m = minimum votes required to be listed in the Top 250 (currently 3000)
• C = the mean vote across the whole report (currently 6.9)

for the Top 250, only votes from regular voters are considered.

I don't know how IMDB chose 3000 as their minimum vote. They could have chosen 1000 or 10000, and the list would have been more or less the same. Maybe they're using "average number of votes after 6 weeks in the box office" or maybe they're using trial and error.

In any case, it doesn't really matter. The formula above is pretty much the standard for normalizing votes on ranking websites, and I'm almost certain Flickrchart uses something similar in the background.

The formula works so well because it "pulls" ratings toward the mean, so ratings above the mean are slightly decreased, ratings below the mean are slightly increased. However, the strength of the pull is inversely proportional to the number of votes a movie has. So movies with few votes are pulled more aggressively toward the mean than movies with lots of votes. Here are two data points to demonstrate the property:

Rank  Movie            Votes            Avg Rating        Weighted Rating
----  -----            -----            ----------        ---------------
219   La Strada        15,000+          8.2               8.0
221   Pirates of the   210,000+         8.0               8.0
      Caribbean 2

Both movies' ratings are pulled down, but the pull on La Strada is more dramatic since it has fewer votes and therefore is not as representative as ratings for PotC.

---

For your specific case, you have two items in a "fight". You should probably design your table as follows:

Items
-----
ItemID (pk)
FightsWon (int)
FightsEngaged (int)

The average rating is FightsWon / FightsEngaged. The weighted rating is calculated using the formula above.

When a user chooses a winner in a fight, increase the winning item's FightsWon field by 1, increase both items FightsEngaged field by 1.

Hope this helps! - Juliet