Python most efficient data structure to hold values and check if a value exist [duplicate]

Question

Say I have millions of string IDs, I want to store them in a variable and check if one ID exists, there are both ways I can think of, list and dict:

Using list

>>> timeit_a = timeit.Timer('"9999999" in a', setup='a = [str(i) for i in range(3000000)]')
>>> timeit_a.timeit(1)
0.06293477199994868

Using dict

>>> timeit_b = timeit.Timer('"9999999" in b', setup='b = {str(i): None for i in range(3000000)}')
>>> timeit_b.timeit(1)
3.860999981952773e-06  # equal to 0.00000386099

As we can see using dict is much much much faster, but I feel creating the dict with bunch of Nones for the sake of just utilizing the hashmap of keys is not very elegant.

Is there a more canonical and more elegant way to do it?

Answer 1

If you have no values, use a set(), not a dict

{str(i) for i in range(30000)}

If you have millions of items, though, maybe offloading to Redis, for example, would be better for an application's memory usage / performance perspective

Answer 2

Definitely use a set. It is like a dict, but without the values, as it is not a mapping but a... set, surprisingly enough.

a = {str(i) for i in range(300000)} # one way of initializing a set
a = set()
for i in range(3000000):
    a.add(str(i)) # another way