Python submodule imports using __init__.py

Question

I'm learning Python, and I can't figure out how imports in __init__.py work.

I understand from the Python tutorial that the __init__.py file initializes a package, and that I can import subpackages here.

I'm doing something wrong, though. Could you explain for me (and for future Python-learners) what I'm doing wrong?

Here's a simplified example of what I'm trying to do.

This is my file structure:

package     __init__.py     test.py     subpackage         __init__.py         hello_world.py 

The contents of hello_world.py:

def do_something():     print "Hello, world!" 

subpackage/__init__.py is empty.

package/__init__.py contains:

import test.submodule.do_something 

And finally, test.py contains:

do_something() 

This is how I attempt to run hello_world.py using OSX terminal and Python 3:

python test.py 

Python then throws the following error:

NameError: name 'do_something' is not defined 

Answer 1

You probably already understand that when you import a module, the interpreter creates a new namespace and executes the code of that module with the new namespace as both the local and global namespace. When the code completes execution, the module name (or the name given in any as clause) is bound to the module object just created within the importing namespace and recorded against its __name__ in sys.modules.

When a qualified name such as package.subpackage.module is imported the first name (package) is imported into the local namespace, then subpackage is imported into package's namespace and finally module is imported into package.subpackage's namespace. Imports using from ... import ... as ... perform the same sequence of operations, but the imported objects are bound directly to names in the importing module's namespace. The fact that the package name isn't bound in your local namespace does not mean it hasn't been imported (as inspection of sys.modules will show).

The __init__.py in a package serves much the same function as a module's .py file. A package, having structure, is written as a directory which can also contain modules (regular .py files) and subdirectories (also containing an __init__.py file) for any sub_packages. When the package is imported a new namespace is created and the package's __init__.py is executed with that namespace as the local and global namespaces. So to answer your problem we can strip your filestore down by omitting the top-level package, which will never be considered by the interpreter when test.py is run as a program. It would then look like this:

test.py subpackage/     __init__.py     hello_world.py 

Now, subpackage is no longer a sub-package, as we have removed the containing package as irrelevant. Focusing on why the do_something name is undefined might help. test.py does not contain any import, and so it's unclear how you are expecting do_something to acquire meaning. You could make it work by using an empty subpackage/__init__.py and then you would write test.py as

from subpackage.hello_world import do_something do_something() 

Alternatively you could use a subpackage/__init__.py that reads

from hello_world import do_something 

which establishes the do_something function inside the subpackage namespace when the package is imported. Then use a test.py that imports the function from the package, like this:

from subpackage import do_something do_something() 

A final alternative with the same __init__.py is to use a test.py that simply imports the (sub)package and then use relative naming to access the required function:

import subpackage subpackage.do_something() 

to gain access to it in your local namespace.

With the empty __init__.py this could also be achieved with a test.py reading

import subpackage.hello_world subpackage.hello_world.do_something() 

or even

from subpackage.hello_world import do_something do_something() 

An empty __init__.py will mean that the top-level package namespace will contain only the names of any subpackages the program imports, which allows you to import only the subpackages you require. This gives you control over the namespace of the top-level package.

While it's perfectly possible to define classes and functions in the __init__.py , a more normal approach is to import things into that namespace from submodules so that importers can just import the top-level package to gain access to its contents with a single-level attribute reference, or even use from to import only the names you specifically want.

Ultimately the best tool to keep you straight is a clear understanding of how import works and what effect its various forms have on the importing namespace.