numpy 1.13 MaskedArrayFutureWarning: setting an item on a masked array which has a shared mask will not copy the mask

Question

I recently upgraded from numpy 1.11 to numpy 1.13 hoping to get rid of this masked array warning, but it's still there:

MaskedArrayFutureWarning: setting an item on a masked array which has a shared mask will not copy the mask and also change the original mask array in the future. Check the NumPy 1.11 release notes for more information.*

Basically my code just changes some values in a masked array, and I'm not sure what this error even means.

I was hoping that the upgrade to numpy 1.13 would resolve this, but I think the error is on my end.

For clarity, I'm running numpy 1.13 despite the warning referencing 1.11:

Python 2.7.12 (default, Nov 19 2016, 06:48:10)

[GCC 5.4.0 20160609] on linux2

Type "help", "copyright", "credits" or "license" for more information.

import numpy as np

np.version

'1.13.0.dev0+Unknown'

Thanks for any help. Cat

Answer 1

This shared mask business is a bit confusing.

The current behavior:

In [150]: x=np.ma.masked_greater(np.arange(8),5)
In [151]: x
Out[151]: 
masked_array(data = [0 1 2 3 4 5 -- --],
             mask = [False False False False False False  True  True],
       fill_value = 999999)
In [152]: y=x[3:6]          # a view
In [153]: y[0]=30           # modify the view
/usr/local/bin/ipython3:1: MaskedArrayFutureWarning: setting an item on a masked array which has a shared mask will not copy the mask and also change the original mask array in the future.
Check the NumPy 1.11 release notes for more information.
  #!/usr/bin/python3

data value change is shared with the source

In [154]: y
Out[154]: 
masked_array(data = [30 4 5],
             mask = [False False False],
       fill_value = 999999)
In [155]: x
Out[155]: 
masked_array(data = [0 1 2 30 4 5 -- --],
             mask = [False False False False False False  True  True],
       fill_value = 999999)

but a mask value change is not:

In [156]: y.mask[0]=True
In [157]: y
Out[157]: 
masked_array(data = [-- 4 5],
             mask = [ True False False],
       fill_value = 999999)
In [158]: x
Out[158]: 
masked_array(data = [0 1 2 30 4 5 -- --],
             mask = [False False False False False False  True  True],
       fill_value = 999999)

Make a new view, and call the unshare method:

In [159]: y=x[3:6]
In [160]: y.unshare_mask()
Out[160]: 
masked_array(data = [30 4 5],
             mask = [False False False],
       fill_value = 999999)
In [161]: y[0]=31
In [162]: y
Out[162]: 
masked_array(data = [31 4 5],
             mask = [False False False],
       fill_value = 999999)
In [163]: x
Out[163]: 
masked_array(data = [0 1 2 31 4 5 -- --],
             mask = [False False False False False False  True  True],
       fill_value = 999999)

This changes the data, without issuing the warning.

The future behavior, without the warning, can be produced with:

In [172]: x=np.ma.masked_greater(np.arange(8),5)
In [174]: y=x[3:6]
In [175]: y._sharedmask=False
In [176]: y[0]=30
In [177]: y.mask[0]=True
In [178]: y
Out[178]: 
masked_array(data = [-- 4 5],
             mask = [ True False False],
       fill_value = 999999)
In [179]: x
Out[179]: 
masked_array(data = [0 1 2 -- 4 5 -- --],
             mask = [False False False  True False False  True  True],
       fill_value = 999999)

The new value and mask appear in both y and x.

The bottom line is - what should happen to the mask in x when you change values in y (data or mask)? Change or not?

=================

Or a case where setting a data value in the view also changes the mask might be clearer:

In [199]: x=np.ma.masked_greater(np.arange(8),5)
In [200]: y=x[4:]
In [201]: y
Out[201]: 
masked_array(data = [4 5 -- --],
             mask = [False False  True  True],
       fill_value = 999999)
In [202]: y[-1]=0
/usr/local/bin/ipython3:1: MaskedArrayFutureWarning: setting an item on a masked array which has a shared mask will not copy the mask and also change the original mask array in the future.
Check the NumPy 1.11 release notes for more information.
  #!/usr/bin/python3
In [203]: y
Out[203]: 
masked_array(data = [4 5 -- 0],
             mask = [False False  True False],
       fill_value = 999999)
In [204]: x
Out[204]: 
masked_array(data = [0 1 2 3 4 5 -- --],
             mask = [False False False False False False  True  True],
       fill_value = 999999)

The last y value was unmasked, but the corresponding x was not (I should have showed the change the x.data). This is the current behavior that you are warned about.

But with the future behavior:

In [205]: y=x[4:]
In [206]: y._sharedmask=False
In [207]: y[-1]=0
In [208]: y
Out[208]: 
masked_array(data = [4 5 -- 0],
             mask = [False False  True False],
       fill_value = 999999)
In [209]: x
Out[209]: 
masked_array(data = [0 1 2 3 4 5 -- 0],
             mask = [False False False False False False  True False],
       fill_value = 999999)

x data and mask were changed along with y.