Spark DataFrame groupBy

Question

I have Spark Java that looked like this. Code pulls data from oracle table using JDBC and displays the groupby output.

DataFrame jdbcDF = sqlContext.read().format("jdbc").options(options).load();
jdbcDF.show();   
jdbcDF.groupBy("VA_HOSTNAME").count().show();

Long ll = jdbcDF.count();
System.out.println("ll="+ll);

When I ran the code, jdbcDF.show(); is working, whereas the groupBy and count are not printing anything and no errors were thrown.

My column name is correct. I tried by printing that column and it worked, but when groupBy it's not working.

Can someone help me with DataFrame output? I am using spark 1.6.3.

Answer 1

You can try

import org.apache.spark.sql.functions.count

jdbcDF.groupBy("VA_HOSTNAME").agg(count("*")).show()