I am trying to filter a DataFrame by keeping only those rows that have a certain string column non-empty.
The operation is the following:
df.filter($"stringColumn" !== "")
My compiler shows that the !== is deprecated since I moved to Spark 2.0.1
How can I check if a string column value is empty in Spark > 2.0?
Use =!=
as a replacement:
df.filter($"stringColumn" =!= "")
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With