How build hadoop sources under windows?

Question

Trying to build hadoop from sources under windows 7 x64. According to instructions Hadoop2OnWindows and BUILDING

I cloned hadoop sources from git, checkout to origin/branch-2.5 (SHA-1: fa3bb675a728105d69614f53abe4339958550adf) Then from windows console I run:

1. set Platform=x64
2. clean install -Pdist,native-win -DskipTests -Dtar

And get error - [ERROR] Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:2.5.0-SNAPSHOT:protoc (compile-protoc) on project hadoop-common: org.apache.maven.plugin.MojoExecutionException: 'protoc --version' did not return a version -> [Help 1]

Any ideas how to resolve this?

Answer 1

The first thing the protoc mojo of the hadoop-maven-plugins does is it uses Java's java.lang.Process class to attempt to execute the following command:

protoc --version

Basically it is doing this to check that the version of Protocol Buffer compiler (protoc) on your system matches the version of the protobuf JAR.

There are 4 outcomes for this:

• it couldn't find the protoc command to execute (it receives an exit value of 127)
• it does not receive the exit value of 127, but also doesn't get a version value returned
• a version is returned but doesn't match that of the protobuf jar
• a version is returned and it does match that of the protobuf jar

You are experiencing the 2nd issue.

If you try running protoc --version from the command line does it work?

As they state in their website, if you have multiple versions of protoc in your system, you can set in your build shell the HADOOP_PROTOC_PATH environment variable to point to the one you want to use for the Hadoop build. If you don't define this environment variable, protoc is looked up in the PATH. Is it available on your PATH?

If you haven't installed the Protocol Buffer compiler you can downloaded it from the following location: https://developers.google.com/protocol-buffers/docs/downloads