Python 3.4 and Pandas 0.15.0
df is a dataframe and col1 is a column. With the code below, I'm checking for the presence of the value 10 and replacing such values with 1000.
df.col1[df.col1 == 10] = 1000
Here's another example. This time, I'm changing values in col2 based on index.
df.col2[df.index == 151] = 500
Both these produce the warning below:
-c:1: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame See the the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
Finally,
cols = ['col1', 'col2', 'col3'] df[cols] = df[cols].applymap(some_function)
This produces a similar warning, with an added suggestion:
Try using .loc[row_indexer,col_indexer] = value instead
I'm not sure I understand the discussion pointed to in the warnings. What would be a better way to write these three lines of code?
Note that the operations worked.
Warnings should never be ignored. If you have ever done data analysis or manipulation with Pandas, it is highly likely that you encounter the SettingWithCopy warning at least once. This warning occurs when we try to do an assignment using chained indexing because chained indexing has inherently unpredictable results.
Pandas, however, can be tricked into allowing duplicate column names. Duplicate column names are a problem if you plan to transfer your data set to another statistical language.
When it comes to selecting rows and columns of a pandas DataFrame, loc and iloc are two commonly used functions. Here is the subtle difference between the two functions: loc selects rows and columns with specific labels. iloc selects rows and columns at specific integer positions.
The issue here is that: df.col1[df.col1 == 10]
returns a copy.
So I would say:
row_index = df.col1 == 10 # then with the form .loc[row_indexer,col_indexer] df.loc[row_index, 'col1'] = 100
Agreed with Paul about 'loc' usage.
For your applymap case you should be able to do this:
cols = ['col1', 'col2', 'col3'] df.loc[:, cols] = df[cols].applymap(some_function)
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With