Not deleting rows as part of function - python

Question

Please keep in mind i am coming from an R background (quite novice as well).

I am trying to create a UDF to format a data.frame df in Python, according to some defined rules. The first part deletes the first 4 rows of the data.frame and the second adds my desired column names. My function looks like this:

def dfFormatF(x):
    #Remove 4 first lines
    x = x.iloc[4:]
    #Assign column headers
    x.columns = ['Name1', 'Name2', 'Name3']

dfFormatF(df)

When i run it like this, its not working (neither dropping the first rows nor renaming). When i remove the x=x.iloc[4:], the second part x.columns = ['Name1', 'Name2', 'Name3'] is working properly and the column names are renamed. Additionally, if i run the removal outside the function, such as:

def dfFormatF(x):
        #Assign column headers
        x.columns = ['Name1', 'Name2', 'Name3']
    df=df.iloc[4:]
    dfFormatF(df)

before i call my function i get the full expected result (first removal of the first rows and then the desired column naming).

Any ideas as to why it is not working as part of the function, but it does outside of it?

Any help is much appreciated.

Thanks in advance.

Answer 1

The issue here is that the changes only inside the scope of dfFormatF(). Once you exit that function, all changes are lost because you do not return the result and you do not assign the result to something in the module-level scope. It's worth taking a step back to understand this in a general sense (this is not a Pandas-specific thing).

Instead, pass your DF to the function, make the transformations you want to that DF, return the result, and then assign that result back to the name you passed to the function.

Note This is a big thing in Pandas. What we emulate here is the inplace=True functionality. There are lots of things you can do to DataFrames and if you don't use inplace=True then those changes will be lost. If you stick with the default inplace=False then you must assign the result back to a variable (with the same or a different name, up to you).

import pandas as pd

starting_df = pd.DataFrame(range(10), columns=['test'])

def dfFormatF(x):
    #Remove 4 first lines
    x = x.iloc[4:]
    #Assign column headers
    x.columns = ['Name1']
    print('Inside the function')
    print(x.head())
    return x

dfFormatF(starting_df)
print('Outside the function')
print(starting_df) # Note, unchanged

# Take 2
starting_df = dfFormatF(starting_df)
print('Reassigning changes back')
print starting_df.head()