How does Python handle subpackages?

Question

Say Ansible was installed by means of "pip install ansible". Right after the install the following import statement succeeds:

from ansible.module_utils.basic import AnsibleModule

Now, a local package named "ansible.module_utils.custom" is created. The directory structure:

 ansible/
   __init__.py
   module_utils/
     __init__.py
     custom/
       __init__.py
       utils.py

As soon as this is put in place the aforementioned import statement fails. Claiming "basic" is undefined. The local package does indeed not declare a "basic" subpackage. Only the installed Ansible library does. It seems Python limited its search to the local package only.

I was under the impression Python would consider the complete system path before giving up on finding code. That it would backtrack out of the local package and finally hit the installed Ansible library.

Is this an incorrect assumption ? If so, is it possible at all to make the local package to coexist with the installed package ?

Answer 1

How Import works

import abc

The first thing Python will do is look up the name abc in sys.modules. This is a cache of all modules that have been previously imported.

If the name isn’t found in the module cache, Python will proceed to search through a list of built-in modules. These are modules that come pre-installed with Python and can be found in the Python Standard Library. If the name still isn’t found in the built-in modules, Python then searches for it in a list of directories defined by sys.path. This list usually includes the current directory, which is searched first.

When Python finds the module, it binds it to a name in the local scope. This means that abc is now defined and can be used in the current file without throwing a NameError.

If the name is never found, you’ll get a ModuleNotFoundError. You can find out more about imports in the Python documentation here!