Why do I need to include sub-packages in setup.py

Question

I have a python package called mltester which contains two sub-packages (actions, dialogs) and a main script ml_tester.py, structured as follows:

+ <ProjectFolder>
+---+ <mltester>
|   +---- <actions>
|   +---- <dialogs>
|   +---- ml_tester.py
|   +---- __init__.py
+---- setup.py

My __init__.py looks as follows:

import actions
import dialogs
import ml_tester

In ml_tester.py I do something like:

from actions import *
from dialogs import *

All works fine when running from eclipse. When doing pip install, the following setup.py works fine:

from setuptools import setup
setup(
    name="MLTester",
    version="1.0",
    packages=["mltester",
              "mltester.actions",
              "mltester.dialogs"],
    install_requires=[
        "matplotlib",
    ],
    entry_points='''
        [console_scripts]
        ml_tester_gui=mltester.ml_tester:main
    '''
)

But when I remove "mltester.actions", "mltester.dialogs" from the list of packages, I now get an error like:

File "/usr/local/lib/python2.7/dist-packages/mltester/__init__.py", line 1, in <module>
    import actions
ImportError: No module named actions

And I don't understand why listing just the containing mltester package is not enough. Of Course I can simply add the packages back, but now I think that I'm missing something more conceptual here.

Answer 1

Because packages do not do any package lookup in subtree. Adding a package to packages will only include the package itself and all direct submodules, but none of the subpackages.

For example, if you have a source tree with package spam containing a module eggs and subpackage bacon:

src
└── spam
    ├── __init__.py
    ├── eggs.py
    └── bacon
        └── __init__.py

Specifying packages=['spam'] will include only spam and spam.eggs, but not spam.bacon, so spam.bacon will not be installed. You have to add it separately to include the complete source codebase: packages=['spam', 'spam.bacon'].

To automate the building of the packages list, setuptools offers a handy function find_packages:

from setuptools import find_packages, setup

setup(
    packages=find_packages(),
    ...
)