Get the last element from Apache Spark SQL split() Function

Question

I want to get the last element from the Array that return from Spark SQL split() function.

split(4:3-2:3-5:4-6:4-5:2,'-')

I know it can get by

split(4:3-2:3-5:4-6:4-5:2,'-')[4]

But i want another way when i don't know the length of the Array . please help me.

Answer 1

You can also use SparkSql Reverse() function on a column after Split(). For example:

SELECT reverse(split(MY_COLUMN,'-'))[0] FROM MY_TABLE

Here [0] gives you the first element of the reversed array, which is the last element of the initial array.

Answer 2

Please check substring_index it should work exactly as you want:

substring_index(lit("1-2-3-4"), "-", -1) // 4