Unable to run UDF on hive server

Question

I'm running hive in thrift server mode. I have a UDF in a jar file that I'm trying to use by running

add jar <path to jar>
create temporary function func_name as 'com.test.udf.UDF_CLASS'

However, when I run the create temporary function command I get an error

Query returned non-zero code: 9, cause: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.FunctionTask

I've checked that the name & path of the jar and that the user running the hive server has all the correct permissions to access the jar. What is weird is if I run hive in CLI mode and run the exact same commands, then everything works. Any ideas on why hive CLI may be behaving differently from the hive thrift server?

Answer 1

After playing with this for awhile I got it to work by putting the .class file into a directory structure that matches it's package, and adding it to the .jar from there. For reference I've included the whole process, including compiling and inserting into hive.

1. I used the UDF example here
2. Compile it: javac -classpath $CLASSPATH Lower.java. Note: the CLASSPATH was defined like this: CLASSPATH=$(ls $HIVE_HOME/lib/hive-serde-*.jar):$(ls $HIVE_HOME/lib/hive-exec-*.jar):$(ls $HADOOP_HOME/hadoop-core-*.jar), as described here.
3. Copy the .class file to a folder com/example/hive/udf/
4. Added it to a jar with this command: jar -cf lower.jar com/example/hive/udf/Lower.class
5. Verify that the package looks right: jar -tf lower.jar. You should see a line like this: com/example/hive/udf/Lower.class.
6. Import the jar into hive. add jar lower.jar; create temporary function my_lower as 'com.example.hive.udf.Lower';

Answer 2

Fixed the problem using the following steps:

1) Place each UDF jar in /usr/lib/hive/auxlib

2) Specify the path to each jar in hive-site.xml for the hive.aux.jars.path property (ex: file:///usr/lib/hive/auxlib/jar1.jar,file:///usr/lib/hive/auxlib/jar2.jar)

3) Create a script to make a thrift request to the hive server to run create temporary function func_name as 'com.test.udf.ClassName' for each UDF after the hive server is started

Edit: For Hive 0.9 no matter what I did, Hiveserver couldn't find jars in the auxlib directory. To get this to work on Hiveserver 0.9 I had to just dump the jar in the directory specified by Hive's classpath.