How to get unique elements from a numpy array containing numpy arrays with different lengths?

Question

I have a nested numpy array - it contains a lot of other numpy sub-arrays, but the sub-arrays have different lengths. The main array arr_main looks something like this:

>>> print(main_arr)
array([[array([3.5525, ..., 4.0138, 4.0139], dtype=float32)],
       [array([3.5525, ..., 4.0138, 4.0139], dtype=float32)],
                                ...
       [array([3.5525, ..., 4.0138, 4.0139], dtype=float32)]],
  dtype=object)

What I want to do is to extract only the unique sub-arrays from the big, main array, so I want to do something like

np.unique(main_arr)

but this results in the error message ValueError: operands could not be broadcast together with shapes (4613,) (4615,). I guess, this is due to some sub-arrays having different lengths.

How can I extract the unique sub-arrays from main_arr? If you know some solution that is not relying on numpy it will be also appreciated! tnx

Answer 1

The numpy unique function on works on 1 dimensional arrays but here's some logic you could deploy to get an array of unique arrays:

import numpy as np

# Create example array of sub arrays
a = np.array([ 
    np.array([1, 2, 3]), np.array([4, 5, 6, 7]), 
    np.array([1, 2, 3]), np.array([4, 5, 6, 7])])
# Build array of unique sub arrays
unique = []
for sub_a in a: 
    if not any([np.array_equal(i, sub_a) for i in unique]): 
        unique.append(sub_a)
unique_array = np.array(unique)