Using mahout and hadoop

Question

I am a newbie trying to understand how will mahout and hadoop be used for collaborative filtering. I m having single node cassandra setup. I want to fetch data from cassandra

Where can I find clear installation steps for hadoop first and then mahout to work with cassandra?

Answer 1

(I think this is the same question you just asked on [email protected]? Copying my answer.)

You may not need Hadoop at all, and if you don't, I'd suggest you not use it for simplicity. It's "necessary evil" to scale past a certain point.

You can have data on Cassandra but you will want to be able to read it into memory. If you can dump as a file, you can use FileDataModel. Or, you can emulate the code in FileDataModel to create one based on Cassandra.

Then, your two needs are easily answered:

1. This is not even a recommendation problem. Just pick an implementation of UserSimilarity, and use it to compare a user to all others, and pick the ones with highest similarity. (Wrapping with CachingUserSimilarity will help a lot.)

2. This is just a recommender problem. Use a GenericUserBasedRecommender with your UserSimilarity and DataModel and you're done.

It of course can get much more complex than this, but this is a fine start point.

If later you use Hadoop, yes you have to set up Hadoop according to its instructions. There is no Mahout "setup". For recommenders, you would look at one of the RecommenderJob classes which invokes the necessary jobs on your Hadoop cluster. You would run it with the "hadoop" command -- again, this is where you'd need to just understand Hadoop.

The book Mahout in Action writes up most of the Mahout Hadoop jobs in some detail.