Proper initialization of sys.path when main script is in a submodule

Question

The first entry of sys.path is the directory of the current script, according to the docs. In the following setup, I would like to change this default. Imagine the following directory structure:

src/
    core/
        stuff/
        tools/
            tool1.py
            tool2.py
    gui/
        morestuff/
        gui.py

The scripts tool*.py and gui.py are intended to be run as scripts, like the following:

python src/core/tools/tool2.py
python src/gui/gui.py

Now all tools import from src.core.stuff, and the GUI needs gui.morestuff. This means that sys.path[0] should point to src/, but it points to src/core/tools/ or src/gui/ by default.

I can adjust sys.path[0] in every script (with a construct like the following, e.g., at the beginning of gui.py):

if __name__ == '__main__':
    if sys.path[0]: sys.path[0] = os.path.dirname(os.path.abspath(sys.path[0]))

However, this is sort of redundant, and it becomes tedious for a mature code base with thousands of scripts. I also know the -m switch:

python -m gui.gui

But this requires the current directory to be src/.

Is there a better way to achieve the desired result, e.g. by modifying the __init__.py files?

EDIT: This is for Python 2.7:

~$ python -V
Python 2.7.3

Answer 1

The only officially approved way to run a script that is in a package is by using the -m flag. While you could run a script directly and try to do sys.path manipulations yourself in each script, it's likely to be a big pain. If you move a script between folders, the logic for rewriting sys.path may also need to be changed to reflect the new location. Even if you get sys.path right, explicit relative imports will not work correctly.

Now, making python -m mypackage.mymodule work requires that either you be in the project's top level folder (src in your case), or for that top level folder to be on the Python search path. Requiring you to be in a specific folder is awkward, and you've said that you don't want that. Getting src into the search path is our goal then.

I think the best approach is to use the PYTHONPATH environment variable to point the interpreter to your project's src folder so that it can find your packages from anywhere.

This solution is simple to set up (the environment variable can be be set automatically in your .profile, .bashrc or some other equivalent place), and will work for any number of scripts. If you move your project, just update your environment settings and you'll be all set, without needing to do any more work for each script.