Numpy: efficient way to extract range of consecutive numbers

Question

Suppose one has a numpy array of floats where all values are in [0,1] e.g.

arr = np.array([
    [0.1,  0.1,  0.1,  0.4,  0.91, 0.81, 0.84], # channel 1 
    [0.81, 0.79, 0.85, 0.1,  0.2,  0.61, 0.91], # channel 2 
    [0.3,  0.1,  0.24, 0.87, 0.62, 1,    0   ], # channel 3
    #...
])

and that one wants to covert this into a binary array. This can be easily done with a cutoff with:

def binary_mask(arr, cutoff=0.5):
  return (arr > cutoff).astype(int)

m = binary_mask(arr)

# array([[0, 0, 0, 0, 1, 1, 1],
#    [1, 1, 1, 0, 0, 1, 1],
#    [0, 0, 0, 1, 1, 1, 0]])

one can get all the indices of the 1s via

for channel in m:
  print(channel.nonzero())

# (array([4, 5, 6]),)
# (array([0, 1, 2, 5, 6]),)
# (array([3, 4, 5]),)

what would be a performant way to the consecutive runs of numbers

e.g.

[ 
    [[4,6]], 
    [[0,2], [5,6]], 
    [[3,5]]
]

a naive approach might be:

def consecutive_integers(arr):

    # indexes of nonzero
    nz = arr.nonzero()[0]

    # storage of "runs"
    runs = []

    # error handle all zero array
    if not len(nz):
        return [[]]

    # run starts with current value
    run_start = nz[0]
    for i in range(len(nz)-1):

        # if run is broken
        if nz[i]+1 != nz[i+1]:
            # store run
            runs.append([run_start, nz[i]])

        # start next run at next number
        run_start = nz[i+1]

    # last run ends with last value
    runs.append([run_start, nz[-1]])

    return runs



print(m)
for runs in [consecutive_integers(c) for c in m]:
    print(runs)



# [[0 0 0 0 1 1 1]
#  [1 1 1 0 0 1 1]
#  [0 0 0 1 1 1 0]]
# 
# [[4, 6]]
# [[0, 2], [5, 6]]
# [[3, 5]]

Answer 1

You can compare consecutive indicators and then use where or flatnonzero

>>> x
array([[0, 0, 0, 0, 1, 1, 1],
       [1, 1, 1, 0, 0, 1, 1],
       [0, 0, 0, 1, 1, 1, 0]])
>>> 
# find switches 0->1 and 1->0
>>> d = np.empty((np.arange(2) + x.shape), bool)
>>> d[:, 0] = x[:, 0]   # a 1 in the first
>>> d[:, -1] = x[:, -1] # or last column counts as a switch
>>> d[:, 1:-1] = x[:, 1:] != x[:, :-1]
>>> 
# find switch indices (of flattened array)
>>> b = np.flatnonzero(d)
# create helper array of row offsets
>>> o = np.arange(0, d.size, d.shape[1])
# split into rows, subtract row offsets and reshape into start, end pairs
>>> result = [(x-y).reshape(-1, 2) for x, y in zip(np.split(b, b.searchsorted(o[1:])), o)]
>>> 
>>> result
[array([[4, 7]]), array([[0, 3],
       [5, 7]]), array([[3, 6]])]

This uses python convention, i.e. right end excluded. If you want right end included use result = [(x-y).reshape(-1, 2) - np.arange(2) for x, y in zip(np.split(b, b.searchsorted(o[1:])), o)] instead.

Answer 2

I would check out this answer: https://stackoverflow.com/a/7353335/1141389

It uses np.split