Python: Hold data in memory after closing program

Question

I have a several huge attributes in my code which takes about 4 mb each. When I start the application in Eclipse, the program has to load all these attributes from files. It takes about 30 seconds. Is there a way to hold these attributes in memory, so I would not have to wait for 30 seconds everytime I run the program?

I think it would be possible in C using pointers.

Thanks for advices

Answer 1

You cannot do it using C pointers. When program exists all it's allocated memory is released. As well as other resources like open files and database connections.

What you can do is set-up some memory cache server. You can use memcached as John Barca suggested, personally I use Redis http://redis.io/ for such things.

However, keep in mind that memcached and redis will only help with file reading times. How many of 4mb attributes do you have? On modern computer loading 4mb file should be instantaneous. Possibly your program spends time parsing the attributes read from the file, in such case you'd better use something like pickle to save and load the objects. You should have some logic in your program that compares the time-stamps on you pickled files and the original data files and load from appropriate source.