Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to implement a Digg-like algorithm?

How to implement a website with a recommendation system similar to stackoverflow/digg/reddit? I.e., users submit content and the website needs to calculate some sort of "hotness" according to how popular the item is. The flow is as follows:

  • Users submit content
  • Other users view and vote on the content (assume 90% of the users only views content and 10% actively votes up or down on content)
  • New content is continuously submitted

How do I implement an algorithm that calculates the "hotness" of a submitted item, preferably in real-time? Are there any best-practices or design patterns?

I would assume that the algorithm takes the following into consideration:

  • When an item was submitted
  • When each vote was cast
  • When the item was viewed

E.g. an item that gets a constant trickle of votes would stay somewhat "hot" constantly while an item that receives a burst of votes when it is first submitted will jump to the top of the "hotness"-list but then fall down as the votes stop coming in.

(I am using a MySQL+PHP but I am interested in general design patterns).

like image 737
Niklas Avatar asked Sep 16 '08 12:09

Niklas


2 Answers

You could use something similar to the Reddit algorithm - the basic principle of which is you compute a value for a post based on the time it was posted and the score. What's neat about the Reddit algorithm is that you only need recompute the value when the score of a post changes. When you want to display your front page, you just get the top n posts from your database based on that score. As time goes on the scores will naturally increase, so you don't have to do any special processing to remove items from the front page.

like image 136
Kyle Cronin Avatar answered Sep 29 '22 12:09

Kyle Cronin


On my own site, I assign each entry a unique integer from a monotonically increasing series (newer posts get higher numbers). Each up vote increases the number by one, and each down vote decreases it by one (you can tweak these values, of course). Then, simply sort by the number to display the 'hottest' entries.

like image 26
Nick Johnson Avatar answered Sep 29 '22 12:09

Nick Johnson