Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Apache Pig UDF Resolution Issue

I've got the following Java code (returns fixed value for testing):

Static.java

package com.company;

import java.io.IOException;
import org.apache.pig.EvalFunc;
import org.apache.pig.data.Tuple;
import org.apache.pig.impl.util.WrappedIOException;

public class Static extends EvalFunc<String>
{
        public String exec(Tuple input) throws IOException
        {
                if (input == null || input.size() == 0)
                    return null;
                try
                {
                        String str = (String)input.get(0);
                        return "5876L";
                }
                catch (Exception e)
                {
                        throw WrappedIOException.wrap("Caught exception processing input row ", e);
                }
        }
}

Built using

javac -Xbootclasspath:/usr/lib/jvm/j2sdk1.6-oracle/jre/lib/rt.jar -source 1.6 -cp `hadoop classpath`:/opt/cloudera/parcels/CDH-4.3.0-1.cdh4.3.0.p0.22/lib/pig/pig-0.11.0-cdh4.3.0.jar Static.java

jar -cf Static.jar Static.class

Apache Pig job

REGISTER /path/to/Static.jar;

loaded = LOAD 'data/' USING com.twitter.elephantbird.pig.store.LzoPigStorage() AS (line:chararray);

loaded = SAMPLE loaded 0.00001;

sized = FOREACH loaded GENERATE com.company.Static(line);

DUMP sized;

Is ran with

# pig -f static.pig -x local

Which errors

[main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1070: Could not resolve com.company.Static using imports: [, org.apache.pig.builtin., org.apache.pig.impl.builtin.]

Changing the line

sized = FOREACH loaded GENERATE com.company.Static(line);

To

sized = FOREACH loaded GENERATE Static(request);

It errors with

[main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2998: Unhandled internal error. Static (wrong name: com/company/Static)
like image 842
Carl Sagan Avatar asked Feb 13 '26 05:02

Carl Sagan


1 Answers

You seem to be missing the required directory structure in the jar. com.company.Static (i.e., the Static.class file) should be located under the com/company directory in the jar. See this other SO question for more details.

For more details, see my response to this other question you posted.

like image 129
cabad Avatar answered Feb 15 '26 20:02

cabad



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!