Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

how to create a small constant relation(table) in pig?

Tags:

apache-pig

is there a way to create a small constant relation(table) in pig? I need to create a relation with only 1 tuple that contains constant values. something along the lines of:

A = LOAD using ConstantLoader('{(1,2,3)}');

thanks, Ido

like image 737
ihadanny Avatar asked Nov 16 '12 09:11

ihadanny


2 Answers

I'm not sure why you would need that but, here's an ugly solution:

A = LOAD 'some/sample/file' ;
B = FOREACH A GENERATE '' ;
C = LIMIT A 1 ;

Now, you can use 'C' as the 'empty relation' that has one empty tuple.

like image 180
Prasoon Joshi Avatar answered Oct 29 '22 17:10

Prasoon Joshi


DEFINE GenerateRelationFromString(string) RETURNS relation {
    temp = LOAD 'somefile';
    tempLimit1 = LIMIT temp 1;
    $relation = FOREACH tempLimit1 GENERATE FLATTEN(TOKENIZE('$string', ','));
};

usage:

fourRows = GenerateRelationFromString('1,2,3,4');
myConstantRelation = FOREACH fourRows GENERATE ( 
CASE $0
    WHEN '1' THEN (1, 'Ivan')
    WHEN '2' THEN (2, 'Boris')
    WHEN '3' THEN (3, 'Vladimir')
    WHEN '4' THEN (4, 'Olga')
END
) as myTuple;

This for sure is hacky, and the right way, in my mind, would be to implement a StringLoader() that would work like this:

fourRows = LOAD '1,2,3,4' USING StringLoader(',');

The argument typically used for file location would just be used as litral string input.

like image 38
Igor Yagolnitser Avatar answered Oct 29 '22 17:10

Igor Yagolnitser