Pandas - Replace values in a DataFrame Based on a Boollean DataFrame

Question

I'm using Pandas v0.20.2 and I have DataFrame, like the following:

df = pd.DataFrame(dict(a=[0,1], b=[3,4], c=[6,7]), 
              index=['spam', 'ham'])
#       a  b  c
# spam  0  3  6
# ham   1  4  7

And I have another DataFrame that is a mask:

mask = pd.DataFrame(dict(a=[True,False], b=[True,True]), 
                index=['spam', 'ham'])
#           a     b
# spam   True  True
# ham   False  True

And I want to set the values in df equal to 999 where it is True in the mask.

I thought that the following would work:

df[mask] = 999

But it doesn't. I get the error below:

ValueError                                Traceback (most recent call last)
<ipython-input-65-503f937859ab> in <module>()
----> 1 df[mask] = 999

/home/gbra/anaconda3/envs/outer_disk/lib/python2.7/site-packages/pandas/core/frame.pyc in __setitem__(self, key, value)
   2326             self._setitem_array(key, value)
   2327         elif isinstance(key, DataFrame):
-> 2328             self._setitem_frame(key, value)
   2329         else:
   2330             # set column

/home/gbra/anaconda3/envs/outer_disk/lib/python2.7/site-packages/pandas/core/frame.pyc in _setitem_frame(self, key, value)
   2364         self._check_inplace_setting(value)
   2365         self._check_setitem_copy()
-> 2366         self._where(-key, value, inplace=True)
   2367 
   2368     def _ensure_valid_index(self, value):

/home/gbra/anaconda3/envs/outer_disk/lib/python2.7/site-packages/pandas/core/generic.pyc in _where(self, cond, other, inplace, axis, level, try_cast, raise_on_error)
   5096             for dt in cond.dtypes:
   5097                 if not is_bool_dtype(dt):
-> 5098                     raise ValueError(msg.format(dtype=dt))
   5099 
   5100         cond = cond.astype(bool, copy=False)

ValueError: Boolean array expected for the condition, not float64

I would appreciate any help on this.

Answer 1

You can reindex the mask to have the same shape as df, and then use df.mask:

df.mask(mask.reindex(df.index, df.columns, fill_value=False), 999)
Out: 
        a    b  c
spam  999  999  6
ham     1  999  7

At that point, regular indexing should also work:

df[mask.reindex(df.index, df.columns, fill_value=False)] = 999

Answer 2

This will do the job:

df = pd.DataFrame(dict(a=[0,1], b=[3,4], c=[6,7]), 
              index=['spam', 'ham'])
mask = pd.DataFrame(dict(a=[True,False], b=[True,True]), 
                index=['spam', 'ham'])
df.iloc[mask] = 999

Then df is

        a   b     c
spam    999 999   6
ham     1   999   7