Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Pig Script without load

I am a newbie to Pig. I am trying to figure out how to define a bag or tuple with hard coded values, without loading data from a file. Every example that I have encountered with starts with:

a = LOAD '/file/name' using PigStorage(',');

or something similar. I just want to create a tuple or bag like this:

a = <1,2,3>;

Is it possible to use hard coded values for testing purposes?

Thanks.

like image 847
user3900607 Avatar asked Aug 01 '14 19:08

user3900607


People also ask

How do you run a Pig script online?

Pig scripts can be run interactively from within Big Data Studio by directly typing into the provided console. Execute complete Pig script. You can execute complete script file loaded in Editor by clicking the “Execute” button.

What is flatten in Pig?

The FLATTEN operator looks like a UDF syntactically, but it is actually an operator that changes the structure of tuples and bags in a way that a UDF cannot. Flatten un-nests tuples as well as bags. The idea is the same, but the operation and result is different for each type of structure.


2 Answers

Unfortunately it is impossible to just create a tuple or bag in Pig like this in the current version (0.15.0).

My suggestion is to create a simple text file with a few values separated by commas, and use the following command:

a = LOAD '/path' using PigStorage(',');

The text file should look something like:

1,2,3

Hopefully in future versions they will release some way to accomplish creating a tuple or bag with a single command.

Good Luck practicing Pig!

like image 93
Bryan Linton Avatar answered Oct 31 '22 16:10

Bryan Linton


As far as I can tell, there is no way to declare hard coded values with PigLatin itself. If you wish to test your scripts, you may want to use UDF. This will let you declare what you want in the language of your choice.

like image 25
merours Avatar answered Oct 31 '22 16:10

merours