Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why I cannot use callUDF method?

I use spark 1.6.1, and code in java. When I use callUDF(), it shows

The method callUDF(String, Column) is undefined for the type PhaseOne

and callUdf() does not work. My codes are as follows:

sqlContext.udf().register("stringToLong", new UDF1<String, Long>() {

        @Override
        public Long call(String arg0) throws Exception {
            // TODO Auto-generated method stub
            IPTypeConvert itc = new IPTypeConvert();
            return itc.stringtoLong(arg0);
        }
    }, DataTypes.LongType);
    DataFrame interDF = initInterDF.withColumn("interIPInt", callUDF("stringToLong", initInterDF.col("interIP")));
like image 497
volity Avatar asked Jul 04 '26 18:07

volity


1 Answers

You must add at the beginning:

import static org.apache.spark.sql.functions.callUDF;

And then use it:

sqlContext.udf().register("stringToLong", new UDF1<String, Long>() {

        @Override
        public Long call(String arg0) throws Exception {
            // TODO Auto-generated method stub
            IPTypeConvert itc = new IPTypeConvert();
            return itc.stringtoLong(arg0);
        }
    }, DataTypes.LongType);
DataFrame interDF = initInterDF.withColumn("interIPInt", callUDF("stringToLong", initInterDF.col("interIP")));
like image 125
T. Gawęda Avatar answered Jul 07 '26 07:07

T. Gawęda



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!