Importing Python modules from different levels of hierarchy

Question

Within the top level of my git repository, I have the following file structure:

miscellaneous Dockerfiles, readme, etc
Code/
    training.py
    data/
        generate.py
        tasksets.py

Sometimes I want to import the generate module from within the tasksets module when I run the tasksets module as a script, so tasksets includes the following import:

import generate

Other times I want to import the tasksets module from within the training module, so training contains the following import:

import tasksets

However, this setup is giving me problems. tasksets can import generate fine when I run tasksets as a script, but throws an error if I import tasksets inside training when I run training as a script (I think because training can't find generate as a script within the default path). I've tried looking at all sorts of other StackOverflow questions and answers, using __init__.py files, relative imports, etc. Currently, my workaround is to use the following lines inside tasksets:

if __name__ == "__main__": import generate
else: from data import generate

But this doesn't feel right (and my IDE don't like it neither). Please can someone explain how to use the right assortment of __init__.py files and import statements such that I can import generate when running tasksets as a script, and also import tasksets when running training as a script?

Answer 1

You better use a classical Python module / package architecture.

projectname/
    __init__.py
    __main__.py
    data/
        __init__.py
        generate.py
        tasksets.py

To use your app, go into projectname/../ directory (one level upper projectname/) and run python -m projectname. This will execute projectname/__main__.py.

---

In __main__.py you will write something like:

from projectname.data import generate
from projectname.data import tasksets

if __name__ == '__main__':
    generate.foo()
    tasksets.bar()

1. You will use absolute import path (starting by your module name and a dot, projectname.)
2. You will import your submodules out of the if __name__ == '__main__'
3. __main__.py will be the only entry-point of your app/script.

In any other file, you will use the same syntax and paths to import other modules:

data/generate.py:

from projectname.data import tasksets

def foo():
    print('SPAM!')
    tasksets.bar()

---

Something I don't really enjoy, but I'm not sure any PEP deny it,

In your projectname/__init__.py file you can write:

from projectname.data import generate
from projectname.data import tasksets

So your two submodules will be imported into your main scope __init__.py, so you are able to import the submodules from this scope, like

data/generate.py:

from projectname import generate

But again, I don't really enjoy this way of doing (because Explicit is better than implicit.)

---

Last but not least,

• You can also use python projectname/__main__.py command, but I still recommend python -m projectname
• You can create a setup.py file using setuptools to "install" your app on your system and simply run projectname command to run it.