Python, looking to extract values from a masked array, then rebuild an array

Question

I’m currently writing something that involves a lot of noise I’m attempting to remove, but in order do this I initially used masks, but the way in which I’m analysing the data breaks using a mask.

The masking is done, I’m looking to extract the data that is not masked, run analysis on this, then rebuild the array with the original order.

array([[3, 0, 3],
       [6, 7, 2],
       [2, 5, 0],
       [2, 1, 4]])

Make Mask

array([[-, -, -],
       [6, 7, 2],
       [-, -, -],
       [2, 1, 4]])

Extract Values

array([[6, 7, 2],
       [2, 1, 4]])

Do analysis

Rebuild Array

array([[-, -, -],
       [6, 7, 2],
       [-, -, -],
       [2, 1, 4]])

I’m hoping for an efficient way of doing this as I’m dealing with 100 million data points. Any suggestions are appreciated.

Answer 1

You could use

masked[~masked.mask] = analyzed.ravel()

to reassign the analyzed values to the masked array.

---

import numpy as np

arr = np.array([[3, 0, 3],
                [6, 7, 2],
                [2, 5, 0],
                [2, 1, 4]])

masked = np.ma.masked_array(arr, mask=False)
masked.mask[::2, None] = True

extracted = np.ma.compress_rows(masked)

analyzed = extracted*10

masked[~masked.mask] = analyzed.ravel()

print(masked)

yields

[[-- -- --]
 [60 70 20]
 [-- -- --]
 [20 10 40]]