Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

What is the closest thing to Apache Hadoop in other languages?

Specifically any Open Source implementations at any degree of usefulness in the following languages:

1) C++

2) Python

3) Ruby

4) C#

like image 627
algorithmicCoder Avatar asked May 21 '11 22:05

algorithmicCoder


People also ask

Is Hadoop a Python or Java?

Hadoop framework is written in Java language, but it is entirely possible for Hadoop programs to be coded in Python or C++ language. This implies that data architects don't have to learn Java if they are familiar with Python.

Which language is used in Hadoop?

The Hadoop framework itself is mostly written in the Java programming language, with some native code in C and command line utilities written as shell scripts.

Is Hadoop a Java framework?

Apache Hadoop is an open source, Java-based software platform that manages data processing and storage for big data applications. The platform works by distributing Hadoop big data and analytics jobs across nodes in a computing cluster, breaking them down into smaller workloads that can be run in parallel.

Is spark better than Hadoop?

Data fragments in Hadoop can be too large and can create bottlenecks. Thus, it is slower than Spark. Spark is much faster as it uses MLib for computations and has in-memory processing. Hadoop has a slower performance as it uses disk for storage and depends upon disk read and write operations.


2 Answers

The german wikipedia has some software examples for each language. I'm translating:

  • Apache Hadoop MapReduce (natively Java, pipes to C++, Python, etc.)
  • disco Open Source Project (Python and Erlang) by Nokia Research Center
  • Greenplum MapReduce by Greenplum (not sure, seems to be C++ though, but with a SQL layer like PIG/Hive).
  • QtConcurrent Open Source C++ MapReduce implementation (but not distributed) by Qt Software
  • Skynet Ruby Map/Reduce Framework
  • Plasma MapReduce (Ocaml) with its own distributed filesystem called "PlasmaFS"

Source

like image 144
Thomas Jungblut Avatar answered Nov 15 '22 14:11

Thomas Jungblut


For Python there is Disco from Nokia: http://discoproject.org/

like image 41
Zanson Avatar answered Nov 15 '22 12:11

Zanson