Error: Failed to create Data Storage while running embedded pig in java

Question

I wrote a simple program to test the embedded pig in java to run in mapreduce mode. The hadoop version in the server I am running is 0.20.2-cdh3u4a, and pig version is 0.10.0-cdh3u4a.

When I try to run in local mode, it runs successfully. But when I try to run in mapreduce mode, it gives me the error.

I run my program using the following commands as shown in http://pig.apache.org/docs/r0.9.1/cont.html#embed-java

javac -cp pig.jar EmbedPigTest.java
javac -cp pig.jar:.:/etc/hadoop/conf EmbedPigTest.java input.txt

My program gives error as:

Exception in thread "main" java.lang.RuntimeException: Failed to create DataStorage
        at org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:75)
        at org.apache.pig.backend.hadoop.datastorage.HDataStorage.<init>(HDataStorage.java:58)
        at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:214)
        at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:134)
        at org.apache.pig.impl.PigContext.connect(PigContext.java:183)
        at org.apache.pig.PigServer.<init>(PigServer.java:226)
        at org.apache.pig.PigServer.<init>(PigServer.java:215)
        at org.apache.pig.PigServer.<init>(PigServer.java:211)
        at org.apache.pig.PigServer.<init>(PigServer.java:207)
        at WordCount.main(EmbedPigTest.java:9)

In some online resources they say that this problem occurs due to different hadoop version. But, I didn't understand what I should do. Suggestions please !!

Answer 1

This is happening because you are linking to the wrong jar, Please see the link below it describes this issue very well.

http://localsteve.wordpress.com/2012/09/30/embedding-pig-for-cdh4-java-apps-fer-realz/