Local scope vs relative imports inside __init__.py

Question

I've noticed that asyncio/init.py from python 3.6 uses the following construct:

from .base_events import *

...

__all__ = (base_events.__all__ + ...)

The base_events symbol is not imported anywhere in the source code, yet the module still contains a local variable for it.

I've checked this behavior with the following code, put into an __init__.py with a dummy test.py next to it:

test = "not a module"
print(test)

from .test import *
print(test)

not a module
<module 'testpy.test' from 'C:\Users\MrM\Desktop\testpy\test.py'>

Which means that the test variable got shadowed after using a star import.

I fiddled with it a bit, and it turns out that it doesn't have to be a star import, but it has to be inside an __init__.py, and it has to be relative. Otherwise the module object is not being assigned anywhere.

Without the assignment, running the above example from a file that isn't an __init__.py will raise a NameError.

Where is this behavior coming from? Has this been outlined in the spec for import system somewhere? What's the reason behind __init__.py having to be special in this way? It's not in the reference, or at least I couldn't find it.

Answer 1

This behavior is defined in The import system documentation section 5.4.2 Submodules

When a submodule is loaded using any mechanism (e.g. importlib APIs, the import or import-from statements, or built-in import()) a binding is placed in the parent module’s namespace to the submodule object. For example, if package spam has a submodule foo, after importing spam.foo, spam will have an attribute foo which is bound to the submodule.

A package namespace includes the namespace created in __init__.py plus extras added by the import system. The why is for namespace consistency.

Given Python’s familiar name binding rules this might seem surprising, but it’s actually a fundamental feature of the import system. The invariant holding is that if you have sys.modules['spam'] and sys.modules['spam.foo'] (as you would after the above import), the latter must appear as the foo attribute of the former.