Pig Conditional Statements

Question

I think I already know the answer to this, but I just wanted to check here before I give up and do something ugly.

I have a query that needs to count total clicks, and also total distinct users. Total clicks would just be this code without the distinct:

report              = FOREACH report GENERATE user, genre, title;
report              = DISTINCT report;
report              = GROUP report BY (genre, title);

My question is essentially: is there any way to write a conditional statement that would skip the DISTINCT step in this process? Pseudo:

report              = FOREACH report GENERATE user, genre, title;
if $report_type == 'users':
    report              = DISTINCT report;
end if
report              = GROUP report BY (genre, title);

I'd rather not have two separate files, and up to this point the only solutions I can find involve using a Python, etc. wrapper to dynamically deal with it. I'd rather keep everything in a simple .pig file, but can't find a way to do it.

Answer 1

One option could be you can try something like this. Can you check with your input?

input:

user1,action,aa
user2,comedy,cc
user3,drama,dd
user1,action,aa
user1,action,aa
user2,comedy,cc

PigScript:

A = LOAD 'input' USING PigStorage(',') AS (user, genre, title);
B = FOREACH A GENERATE user, genre, title;
C = GROUP B BY (genre, title);
D = FOREACH C {
                noDistValue = FOREACH B GENERATE user,genre,title;
                distValue =  DISTINCT B;
                GENERATE $0 AS grp,noDistValue,distValue;
              }
E = FOREACH D GENERATE grp,(('$report_type' == 'users')?distValue:noDistValue) AS mybag;
DUMP E;

Output1:
>>pig -x local -param "report_type=users" test.pig

((action,aa),{(user1,action,aa)})
((comedy,cc),{(user2,comedy,cc)})
((drama,dd),{(user3,drama,dd)})

Output2:
>>pig -x local -param "report_type=nonusers" test.pig

((action,aa),{(user1,action,aa),(user1,action,aa),(user1,action,aa)})
((comedy,cc),{(user2,comedy,cc),(user2,comedy,cc)})
((drama,dd),{(user3,drama,dd)})

In case if you want to calculate the Count then project the relation E and also you can modify the above script according to your need.