Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Which is the best scheduler for HADOOP. oozie or cron?

Tags:

oozie

Can anyone please suggest which is best suited scheduler for Hadoop. If it is oozie. How is oozie different from cron jobs.

like image 348
jugal bhatt Avatar asked Apr 15 '15 10:04

jugal bhatt


1 Answers

Oozie is the best option.

Oozie Coordinator allows triggering actions when files arrive at HDFS. This will be challenging to implement anywhere else.

Oozie gets callbacks from MapReduce jobs so it knows when they finish and whether they hang without expensive polling. No other workflow manager can do this.

There are some benefits over crontab or any other, pointing some links

https://prodlife.wordpress.com/2013/12/09/why-oozie/

like image 75
Sravan K Reddy Avatar answered Sep 28 '22 06:09

Sravan K Reddy