Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Hadoop JAR command - Setting java.library.path

I'm trying to run a java program on Hadoop cluster. Here's the command-

export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/usr/local/lib/*:/home/rgupta/bdAnalytics/lib/*
hadoop jar $jarpath bigdat.twitter.queue.TweetOMQSub >  $logsFldr/subsHdpOMQ_$1.log 2>&1 &
#java -Djava.library.path=/usr/local/lib -classpath  class/:lib/:lib/jzmq-2.1.3.jar  bigdat.twitter.queue.TweetOMQSub > log/subsFilterOMQ_$1.log 2>&1 &

This throws following error -

Exception in thread "main" java.lang.UnsatisfiedLinkError: no jzmq in java.library.path

If I use the Java native command above, it works fine. Also, the hadoop node where I m trying to test it, does have the necessary jzmq jars under /usr/local/lib directory. Is there a way I can set java.library.path to Hadoop JAR command. Please suggest how can I fix this.

like image 750
user2508012 Avatar asked Jan 30 '26 16:01

user2508012


1 Answers

sorry misread your question so editing:

you should be able to use the libjars option

In your case:

HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/usr/local/lib/:/home/rgupta/bdAnalytics/lib/

hadoop jar $jarpath bigdat.twitter.queue.TweetOMQSub -libjars /usr/local/lib ...
like image 124
Engineiro Avatar answered Feb 03 '26 09:02

Engineiro



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!