Python imports - ModuleNotFoundError: No module named X

Question

I have read probably all of the posts on here regarding imports and I still cannot figure out what is going on with the imports, I have spent hours trying to get a very simple example working and am literally pulling my hair out.

I am using python 3.7 and pycharm but I am running my code from the commandline, for the unit tests I am using pytest.

My project structure is:

my_message_validator/
    __init__.py

    module_1/
        __init.py__
        foo.py

    module_2/
        __init.py__
        bar.py
        baz.py

    module_3
        context.py
        test_all.py

module_1.init.py

from module_1 import foo

module_2.init.py

# For some reason pycharm doesnt complain when I use '.' but if I use module_2 it does
from . import bar, baz

If I try to run my code or my tests from the commandline no matter how I move things around I seem to get either ModuleNotFoundError: No module named, when I have managed to get the tests working I still cannot run my code on its own from the commandline.

How can I import module_1 into module_2 and are these actually packages? I am coming from java and find the imports a lot easier to understand, I am finding the python importing very confusing...

Also how can I can then import whatever I need into my test module\package\folders context.py?

Currently the test context look like:

import os
import sys

# Is this needed as it doesnt seem to do anything?
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))

from module_1.foo import Foo

from module_2 import bar, baz

In test_all.py I am trying to import from the context file like this:

from .context import bar,baz
from .context import Foo

# Calling in test like
Foo.load_file(file)

bar.method_one()
baz.method_two()

Do I need all the __init.py__ files and what should I be putting in them to make my methods and classes public\exposed? I would like this entire package to be reusable so want to be able to treat it like a jar file in java.

Any help would be much appreciated as it seems everytime I change something I get an error in a different place, python seems so much more complicated than java right now.

Answer 1

First, do not use relative imports (with .), as it is known for causing multiple issues. Always write your imports relative to the root of your project. For example, you did it well for from module_1.foo import Foo. You should also do it in test_all.py and context.py. Moreover, after using relative imports, the __init__.py files can be left empty in your case.

Most likely, the Python interpreter cannot find your modules because the PYTHONPATH environment variable does not contain the root of your project. If you run export PYTHONPATH="YOUR_PROJECT_ROOT_ABSOLUTE_PATH:$PYTHONPATH" before your script, it should run as expected. To make sure this variable is set all the time, you can add the export statement to your shell profile file (e.g. .bashrc or .bash_profile).

After chatting with the author, it turns out there was a fourth issue. It was a name collision like the one in this other question. In his project directory, module_1 was actually called foo like its child foo.py, which confused the interpreter.