Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How do I make Python make all identical strings use the same memory? [duplicate]

Possible Duplicate:
What does python intern do, and when should it be used?

I am working with a program in python that has to correlate on an array with millions of string objects. I've discovered that if they all come from the same quoted string, each additional "string" is just a reference to the first, master string. However if the strings are read from a file, and if the strings are all equal, each one still requires new memory allocation.

That is, this takes about 14meg of storage:

a = ["foo" for a in range(0,1000000)]

While this requires more than 65meg of storage:

a = ["foo".replace("o","1") for a in range(0,1000000)]

Now I can make the memory take much less space with this:

s = {"f11":"f11"}
a = [s["foo".replace("o","1")] for a in range(0,1000000)]

But that seems silly. Is there an easier way to do this?

like image 407
vy32 Avatar asked Aug 05 '12 17:08

vy32


1 Answers

just do an intern(), which tells Python to store and take the string from memory:

a = [intern("foo".replace("o","1")) for a in range(0,1000000)]

This also results around 18MB, same as in the first example.

Also note the comment below, if you use python3. Thx @Abe Karplus

like image 86
erikbstack Avatar answered Sep 23 '22 02:09

erikbstack