When importing, why is the entire module loaded even when only specific portions are specified?

Question

I'm trying to get my programs memory footprint under control. I thought I'd start with the imports, as I'm only using 3-4 functions out of the rather large PyObjC library. However, I was a little surprised to see that importing specific pieces of a larger module had zero bearing on what was actually loaded into memory.

Memory Profiler output

loading the entire Quartz.CoreGraphics library on OSX:

Line #    Mem usage    Increment   Line Contents
================================================
    77                             @profile 
    78     7.953 MB     0.000 MB   def test_import_all():
    79    26.734 MB    18.781 MB    import Quartz.CoreGraphics as CG

It pulls in the entire library at almost 19MB.

Trying to only pull in what I need gives the same 19MB results:

Line #    Mem usage    Increment   Line Contents
================================================
    82                             @profile
    83     7.941 MB     0.000 MB   def test_import_some():
    84    26.727 MB    18.785 MB    from Quartz.CoreGraphics import CGImageGetWidth 

So, it seems that specific imports have no bearing on what actually gets loaded.

Needing only a small handful of functions from an otherwise huge module seems like it would be a common use-case. Is there any way to load only what I need from a module into memory, or is this just the consequence of using external libraries?

Answer 1

That's just how module loading works. The runtime maintains a collection of loaded modules, so the whole module is reachable even if you've only imported a few symbols. This has two desirable consequences:

• future imports of the same module are fast.
• any code at module-level that has side-effects, is executed only once (barring a module reload) instead of being executed a variable number of times depending what bits of the module are imported in what places.

It's also fairly inevitable if you consider that any function in the module can access the module namespace, either directly by using names from it or indirectly via globals(), sys.modules[__name__], eval, or whatever. So barring some clever optimization to prove that particular functions don't do this (which Python implementations don't generally bother with), the whole module namespace has to be in memory.