Rails with Hadoop

Question

I will have huge data to store like corers for resumes and would like to consider hadoop as back end . Is this possible to use it with ruby on rails or just need to consider the redis or mongodb.

any pointer to using rails with hadoop would be useful. Or gem that's make the life easy.

Answer 1

Phil Whelan has a great writeup introducing map reduce with ruby using hadoop: http://www.bigfastblog.com/map-reduce-with-ruby-using-hadoop

There is a nice little gem out there (Rubydoop) that makes writing hadoop jobs in ruby nice and easy (with jRuby): https://github.com/iconara/rubydoop

If you need to get more technical, and get into real-time streaming, I've been seeing a lot of use of Casandra + Hadoop + Storm: http://www.slideshare.net/boneill42/ruby-on-big-dag