Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

how to include external jar file using PIG

When I run a mapreduce job using hadoop command, I use -libjars to setup my jar to the cache and the classpath. How to do something like this in PIG?

like image 249
root1982 Avatar asked May 03 '12 01:05

root1982


People also ask

How can we run the local file system commands in pig?

Local Mode - To run Pig in local mode, you need access to a single machine; all files are installed and run using your local host and file system. Specify local mode using the -x flag (pig -x local). Tez Local Mode - To run Pig in tez local mode.

How does pig display data?

Now load the data from the file student_data. txt into Pig by executing the following Pig Latin statement in the Grunt shell. grunt> student = LOAD 'hdfs://localhost:9000/pig_data/student_data.txt' USING PigStorage(',') as ( id:int, firstname:chararray, lastname:chararray, phone:chararray, city:chararray );


1 Answers

There are two ways to add external jars to Pig environment.

  1. Use "-Dpig.additional.jars" to start Pig

    pig -Dpig.additional.jars=/local/path/to/your.jar

  2. Use "register" command in Pig scripts or grunt

    register /local/path/to/your.jar;

You can use any one according to your requirement.

like image 132
2 revs Avatar answered Sep 28 '22 20:09

2 revs