Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How can I iterate through a column of a spark dataframe and access the values in it one by one?

I have spark dataframe Here it is

I would like to fetch the values of a column one by one and need to assign it to some variable?How can it be done in pyspark.Sorry I am a newbie to spark as well as stackoverflow.Please forgive the lack of clarity in question

like image 418
Fasty Avatar asked Oct 17 '22 10:10

Fasty


1 Answers

col1=df.select(df.column_of_df).collect()
list1=[str(i[0]) for i in col1]
#after this we can iterate through list (list1 in this case)
like image 109
Avinash Avatar answered Oct 21 '22 00:10

Avinash