Apache Spark and node.js

Question

I would like to call node.js program from spark-submit, using Spark API within JavaScript. Is there any implementation of this API in JavaScript?

I know one can call external mapper and reducer (written in JavaScript) from Hadoop, but I want to be able to run the complete program as one job from Spark.

Answer 1

There is a new project called EclairJS that is providing a Spark API from Node and JavaScript, see https://github.com/EclairJS/eclairjs-node.

Answer 2

I don't believe there is a current JavaScript spark wrapper, however I'd check out the information on the Spark pipe interface, this will allow you to use your JavaScript code from within Spark. Best of luck with your Spark & Javascript adventures (sadly there was an april fools joke about Spark & ScalaJS which makes searching for information on Spark + Javascript a little bit tricky).