How to use global variable in pyspark function

Question

Firstly I have two variable at begining of code.

numericColumnNames = []
categoricalColumnsNames = [];

Then in main method , I assign value to those values

def main():
  #clickRDD = sc.textFile("s3a://wer-display-ads/day_0_1000.csv"); 
  clickRDD = sc.textFile("data/day_0_1000.csv");
  numericColumnNames , categoricalColumnsNames = getColumnStructure();

Then When I want to use those variables in following function , Those variables are not updated and are empty

def dataToVectorForLinear(clickDF):
  print (categoricalColumnsNames) ## why this list is empty 
  clickDF = oneHotEncoding(clickDF,categoricalColumnsNames)

Unfortunetly I can't find the problem? Thanks for your help

Answer 1

Just re-initialize them inside the function 'global` keyword like this

def main():

    global numericColumnNames
    global categoricalColumnsNames     

    clickRDD = sc.textFile("data/day_0_1000.csv");
    numericColumnNames , categoricalColumnsNames = getColumnStructure();

Similarly

def dataToVectorForLinear(clickDF):

    global categoricalColumnsNames
    print (categoricalColumnsNames) 
    clickDF = oneHotEncoding(clickDF,categoricalColumnsNames)

Reference:

• Global and local variables in Python