Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python: Memory cost of importing a module

The memory cost obviously depends on exactly how large a module is, but I'm only looking for a general answer: Is it generally expensive or cheap to import a module in Python? If I have a few tens of small scripts that potentially stay in memory for the whole duration of the application, how much will that hog the memory?

like image 342
Paul Manta Avatar asked Jul 25 '11 06:07

Paul Manta


People also ask

What happens when a Python module is imported?

When a module is first imported, Python searches for the module and if found, it creates a module object 1, initializing it. If the named module cannot be found, a ModuleNotFoundError is raised. Python implements various strategies to search for the named module when the import machinery is invoked.

How many times does a module get loaded when imported multiple times?

A module code is evaluated only the first time when imported. If the same module is imported into multiple other modules, its code is executed only once, upon the first import. Then its exports are given to all further importers. The one-time evaluation has important consequences, that we should be aware of.

How many times does a module gets loaded when imported multiple times in Python?

The rules are quite simple: the same module is evaluated only once, in other words, the module-level scope is executed just once. If the module, once evaluated, is imported again, it's second evaluation is skipped and the resolved already exports are used.

Does Python import module multiple times?

In fact, Python only loads the module when it is imported in the first file and all subsequent files simply set the name to refer to the already loaded module. While it is possible to override this behavior so that each file has its own copy of the module, it is generally not recommended.


1 Answers

It sounds like you aren't worried about time cost (good; that would be silly, since modules are only imported once) but memory cost. I put it to you: if you need all the functionality in these modules, then how exactly do you plan to avoid having them all in memory? Might as well just import things in the most logical way.

That said, in Python import is a statement and not some kind of preprocessor directive (or similar), and so you can delay the import of a module until you actually need its contents, by simply arranging for the statement to run at the appropriate time. This can be a meaningful optimization in some particularly large or complex projects, or at least make tricky things possible. Remember, the Python compiler doesn't try to check if there is a foo when you refer to foo.bar; every name is looked up at run-time. Duck-typing comes into play here; the foo.bar code doesn't care whether foo is the name of a module, class or object. (That's basically because modules and classes are objects, but I digress...)

like image 181
Karl Knechtel Avatar answered Nov 05 '22 16:11

Karl Knechtel