Spark Data frame search column starting with a string

Question

I have a requirement to filter a data frame based on a condition that a column value should starts with a predefined string.

I am trying following:

 val domainConfigJSON = sqlContext.read
    .jdbc(url, "CONFIG", prop)
    .select("DID", "CONF", "KEY").filter("key like 'config.*'")

And getting exception:

Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'KEY = 'config.*'' at line 1

Using spark: 1.6.1

Answer 1

You can use the startsWith function present in Column class.

myDataFrame.filter(col("columnName").startswith("PREFIX"))