Import local package from local script

Question

I am having this project structure:

.
├── README.md
├── my_project
│   ├── __init__.py
│   ├── config.py
│   ├── integrations
│   │   ├── __init__.py
│   │   └── google_places.py
├── requirements.txt
└── scripts
    └── my_script.py

In my_script.py I have:

from my_project.integrations.google_places import GooglePlaces

However I get the following error when running python scripts/my_script.py:

ModuleNotFoundError: No module named 'my_project'

Locally I can pip install -e . my package (by adding a setup.py to the mix), however since I need to run this from GitHub Actions it feels it doesn't make sense to pursue the same avenue.

I've been reading about manipulating sys.path, but it seems a bit hackish.

Since this is a fairly common setup (I guess), what's the recommended way of fixing the above error? I can alter the project structure if it would make sense.

Answer 1

This also surprised me, but after reading this pretty comprehensive answer https://stackoverflow.com/a/54613085/6180150, and into the linked python docs about the module search path in python, I think you have to adjust the PYTHONPATH. The problem is that the two python files reside in two separate packages, namely scripts and my_project, so my_script.py is not able to find the google_places.py, because the PYTHONPATH only contains the current package and not the parent folder and python will only search following directories (see the aforementioned search path docs):

• The directory containing the input script (or the current directory when no file is specified).
• PYTHONPATH (a list of directory names, with the same syntax as the shell variable PATH).
• The installation-dependent default.

Also in this comment: Python: 'ModuleNotFoundError' when trying to import module from imported package the replier of above answer explains nicely that adjusting the PYTHONPATH is not really a hacky solution (although I would have also felt so), but the documented solution according to python, too.

If possible, in case I were in your shoes, I think I would move the my_script.py into the my_project package, like follows.

.
├── README.md
├── my_project
│   ├── __init__.py
│   ├── config.py
│   ├── integrations
│   │   ├── __init__.py
│   │   └── google_places.py
│   └── my_script.py
└── requirements.txt

Then you can adapt your import to:

from integrations.google_places import GooglePlaces

---

EDIT:

If you like to use the following file tree structure (or even the original should work).

.
├── README.md
├── my_project
│   ├── __init__.py
│   ├── config.py
│   ├── integrations
│   │   ├── __init__.py
│   │   └── google_places.py
│   └── scripts
│       └── my_script.py
└── requirements.txt

You can just modify your sys.path like this and then use the original import statement:

import sys
import os

sys.path.insert(0, os.getcwd())
from my_project.integrations.google_places import GooglePlaces