Possible Duplicate:
What does python intern do, and when should it be used?
I am working with a program in python that has to correlate on an array with millions of string objects. I've discovered that if they all come from the same quoted string, each additional "string" is just a reference to the first, master string. However if the strings are read from a file, and if the strings are all equal, each one still requires new memory allocation.
That is, this takes about 14meg of storage:
a = ["foo" for a in range(0,1000000)]
While this requires more than 65meg of storage:
a = ["foo".replace("o","1") for a in range(0,1000000)]
Now I can make the memory take much less space with this:
s = {"f11":"f11"}
a = [s["foo".replace("o","1")] for a in range(0,1000000)]
But that seems silly. Is there an easier way to do this?
just do an intern()
, which tells Python to store and take the string from memory:
a = [intern("foo".replace("o","1")) for a in range(0,1000000)]
This also results around 18MB, same as in the first example.
Also note the comment below, if you use python3. Thx @Abe Karplus
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With