I'm using a dictionary to hold a large number of objects, and have a string name for each of them. To be specific, here is my code:
from itertools import product
for (i,j,k) in product(range(N),range(M),range(K)):
var_name='x_'+'_'+str(i)+str(j)+'_'+str(k)
var_dict[var_name] = f(var_name,other_params)
print len(var_dict)
f(...)
returns an object. In my code N=363, M=500, and K=2. So I expect 363000 entries in the dictionary. But when I check the length of var_dict
, it is 330860!
(Pdb) len(var_dict)
330860
Here are my questions:
Is there any explanation for that? E.g. is there any limit for the number of items that built-in hash table of python can address?
What can I do to solve this problem?
The problem is here:
str(i)+str(j)
This does not produce unique identifiers. For example, the value set when i=1
and j=11
will be overwritten by the value set when i=11
and j=1
(there are many more instances as well).
You can fix the problem by inserting some delimiter character between the two numbers (such as an underscore like you have between j
and k
).
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With