Python imports structure

Question

I want to have this structure for my project:

requirements.txt
README.md
.gitignore
project/
    __init__.py
    project.py
    core/
        __init__.py
        base.py
    engines/
        __init__.py
        engine1.py
        engine2.py
    utils/
        __init__.py
        refine_data.py
        whatever.py

The application is run from project/project.py. However, I constantly get import errors when using relative or absolute imports.

Both engines need to import from project.core.base, the utils need to import from project.core.base as well, and project.py (the main file ran) needs to be able to import from engines.

Absolute imports don't work:

# engines/engine1.py
from project.core.base import MyBaseClass

which gives the error:

ImportError: No module named project.core.base

But if I try a relative import instead

# engines/engine1.py
from ..core.base import MyBaseClass

I get:

ValueError: Attempted relative import beyond toplevel package

I've seen other projects on Github structured similarly, but this seems to cause all sorts of problems. How do I get this to work?

Answer 1

Take a look at your sys.path. It's likely that the top project directory is in the python path, and it sees your sub-packages (ie. utils, engines, etc.) as separate packages, which is why it's giving you an error that you're trying to import from outside your package when doing relative imports, and absolute imports don't work because it can't find the top project directory because it's not under any of the python paths.

The directory above the top project directory is what needs to be added to the python path.

Ex.

/path/is/here/project/core/...

# Add this to the PYTHONPATH
/path/is/here  

Answer 2

Try to use these imports:

engine1.py:

from core import base

refine_data.py:

from core import base

project.py

from engines import engine1

if you use pycharm mark project directory as sources root and then try to run project.py. If you don't use pycharm you can run project.py by going to project directory and running command:

python project.py