I have some relatively simple code that I'm struggling to put together. I have a CSV that I've read into a dataframe. The CSV is panel data (i.e., unique company and year observations for each row). I have two columns that I want to perform a function on and then I want to create new variables based on the output of the function.
Here's what I have so far with code:
#Loop through rows in a CSV file
for index, rows in df.iterrows():
#Start at column 6 and go to the end of the file
for row in rows[6:]:
data = perform_function1( row )
output = perform_function2(data)
df.ix[index, 'new_variable'] = output
print output
I want this code to iterate starting in column 6 and then going to the end of the file (e.g., I have two columns I want to perform the function on Column6 and Column7) and then create new columns based on the functions that were performed (e.g., Output6 and Output7). The code above returns the output for Column7, but I can't figure out how to create a variable that allows me to capture the outputs from both columns (i.e., a new variable that isn't overwritten by loop). I searched Stackoverflow and didn't see anything that immediately related to my question (maybe because I'm too big of a noob?). I would really appreciate your help.
Thanks,
TT
P.S. I'm not sure if I've provided enough detail. Please let me know if I need to provide more.
By use + operator simply you can combine/merge two or multiple text/string columns in pandas DataFrame. Note that when you apply + operator on numeric columns it actually does addition instead of concatenation.
In pandas you can add/append a new column to the existing DataFrame using DataFrame. insert() method, this method updates the existing DataFrame with a new column. DataFrame. assign() is also used to insert a new column however, this method returns a new Dataframe after adding a new column.
Using assign() assign() method can be used when you need to insert multiple new columns in a DataFrame, when you need to ignore the index of the column to be added or when you need to overwrite the values of an existing columns.
Operating iteratively doesn't take advantage of Pandas' capabilities. Pandas' strength is in applying operations efficiently across the whole dataframe, rather than in iterating row by row. It's great for a task like this where you want to chain a few functions across your data. You should be able to accomplish your whole task in a single line.
df["new_variable"] = df.ix[6:].apply(perform_function1).apply(perform_function2)
perform_function1
will be applied to each row, and perform_function2
will be applied to the results of the first function.
If you want to apply function to certain columns in a dataframe
# Get the Series
colmun6 = df.ix[:, 5]
# perform_function1 applied to each row
output6 = column6.apply(perform_function1)
df["new_variable"] = output6
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With