Pandas drop duplicates on elements made of lists

Question

Say my dataframe is:

df = pandas.DataFrame([[[1,0]],[[0,0]],[[1,0]]])

which yields:

        0
0  [1, 0]
1  [0, 0]
2  [1, 0]

I want to drop duplicates, and only get elements [1,0] and [0,0], if I write:

df.drop_duplicates()

I get the following error: TypeError: unhashable type: 'list'

How can I call drop_duplicates()?

More in general:

df = pandas.DataFrame([[[1,0],"a"],[[0,0],"b"],[[1,0],"c"]], columns=["list", "letter"])

And I want to call df["list"].drop_duplicates(), so drop_duplicates applies to a Series and not a dataframe?

Answer 1

I tried the other answers but they didn't solve what I needed (large dataframe with multiple list columns).

I solved it this way:

df = df[~df.astype(str).duplicated()]

Answer 2

You can use numpy.unique() function:

>>> df = pandas.DataFrame([[[1,0]],[[0,0]],[[1,0]]])
>>> pandas.DataFrame(np.unique(df), columns=df.columns)
        0
0  [0, 0]
1  [1, 0]

If you want to preserve the order checkout: numpy.unique with order preserved