Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Not a Valid Jar When Running Hadoop Job

Tags:

hadoop

I want to run WordCount Example.

In eclipse it run correctly. In output folder the output file is present.

I made a jar file of WordCount and want to run it through command

hadoop jar WordCount.jar /Projects/input /Projects/output

it gives me error

Not a valid JAR: /Projects/WordCount.jar

result of hdfs dfs -ls /Projects

Found 3 items
-rw-r--r-- 1 hduser supergroup 3418 2014-11-02 15:38 /Projects/WordCount.jar
drwxr-xr-x - hduser supergroup 0 2014-11-02 14:13 /Projects/input
drwxr-xr-x - hduser supergroup 0 2014-11-02 14:16 /Projects/output

it gives me same error on this also

  hadoop jar /Projects/WordCount.jar wordPackage.WordCount  /Projects/input /Projects/output

 Not a valid JAR: /Projects/WordCount.jar

how to solve this error.

I have run tvf command it gives this output

 jar -tvf /home/hduser/Desktop/Files/WordCount.jar

  60 Sun Nov 02 16:10:10 PKT 2014 META-INF/MANIFEST.MF
 1895 Sun Nov 02 14:02:38 PKT 2014 wordPackage/WordCount.class
 1295 Sun Nov 02 14:02:38 PKT 2014 wordPackage/WordCount.java
 2388 Sun Nov 02 14:02:06 PKT 2014 wordPackage/WordReducer.class
 707 Sun Nov 02 14:02:06 PKT 2014 wordPackage/WordReducer.java
 2203 Sun Nov 02 14:02:08 PKT 2014 wordPackage/WordMapper.class
 713 Sun Nov 02 14:02:06 PKT 2014 wordPackage/WordMapper.java
 16424 Sun Nov 02 13:50:00 PKT 2014 .classpath
 420 Sun Nov 02 13:50:00 PKT 2014 .project
like image 284
Muhammad Danish Khan Avatar asked Nov 02 '14 11:11

Muhammad Danish Khan


People also ask

What is the command used to run the Hadoop jar file?

jar. Runs a jar file. Users can bundle their Map Reduce code in a jar file and execute it using this command.

Where is Hadoop Common jar?

Maven Repository: org. apache. hadoop » hadoop-common.


1 Answers

You cannot keep the jar in HDFS when executing the same using hadoop command, Jar should be available in the local path

If the jar is not runnable try the following (Need to specify the package.mainclass)

hadoop jar /home/hduser/Desktop/Files/WordCount.jar wordPackage.WordCount /Projects/input /Projects/output

If the jar is runnable following can be used

hadoop jar /home/hduser/Desktop/Files/WordCount.jar /Projects/input /Projects/output

If the issue still persists, you need to rebuild this jar(WordCount.jar) in eclipse again

like image 82
SachinJ Avatar answered Nov 10 '22 04:11

SachinJ