Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python folder structure for project directory and easy import

My team has a folder of several small projects in python3. Amongst them, we have a utility folder with several utility functions, that are used throughout the projects. But the way to import it is very uncomfortable. This is the structure we use:

temp_projects
    util
        storage.py
        geometry.py
    project1
        project1.py
    project2
        project2.py

The problem is that the import in the projects looks terrible:

sys.path.insert(1, os.path.join(sys.path[0], '..'))
import util.geometry

util.geometry.rotate_coordinates(....)

Also, pycharm and other tools are having trouble understanding it and to supply completion.

Is there some neater way to do that?

Edit: All of the projects and utils are very much work-in-progress and are modified often, so I'm looking for something as flexible and comfortable as possible

like image 205
user972014 Avatar asked Jun 12 '18 06:06

user972014


People also ask

Is it easy to structure a Python project?

”. Thanks to the way imports and modules are handled in Python, it is relatively easy to structure a Python project. Easy, here, means that you do not have many constraints and that the module importing model is easy to grasp.

How to import modules from one folder to another in Python?

We can do this using various ways. These ways are discussed below in detail. We can use sys.path to add the path of the new different folder (the folder from where we want to import the modules) to the system path so that python can also look for the module in that directory if it doesn’t found the module in its current directory.

Do Your Python code files belong in a separate subdirectory?

That's right... our Python code files actually belong in a separate subdirectory! This is very important, as our repository's root directory is going to get mighty cluttered with build files, packaging scripts, virtual environments, and all manner of other things that aren't actually part of the source code.

How to import geometry from another folder in Python?

According to Importing files from different folder adding a __init__.py in the util folder will cause python to treat it as a package. Another thing you can do is use import util.geometry as geometry and then you could use geometry.rotate_coordinates (...) which also improves readability. Use importlib.


2 Answers

PYTHONPATH environment variable might be a way to go. Just set it to projects folder location:

PYTHONPATH=/somepath/temp_projects

and you'll be able to use util as follows:

import util.geometry

util.geometry.rotate_coordinates(....)

Also this will be recognized by PyCharm automatically.

like image 180
Vader Avatar answered Oct 24 '22 22:10

Vader


I believe the correct route would be completely different than what you are doing right now. Each project should be stored in a different Git repository, share modules should be added as git submodules. Once those projects will get larger and more complex (and they probably will), it would be easier to manage them separately.

In short

Projects structure should be:

Project_1
  |- utils <submodule>
       |- storage.py
       |- geometry.py
  |- main.py

Project_2
  |- utils <submodule>
       |- storage.py
       |- geometry.py
  |- main.py

Working with submodules

### Adding a submodule to an existing git directory
git submodule add <git@github ...> <optional path/to/submodule>

### Pulling latest version from master branch for all submodules
git submodule update --recursive --remote

### Removing submodule from project
# Remove the submodule entry from .git/config
git submodule deinit -f path/to/submodule

# Remove the submodule directory from the project's .git/modules directory
rm -rf .git/modules/path/to/submodule

# Remove the entry in .gitmodules and remove the submodule directory located at path/to/submodule
git rm -f path/to/submodule

Further reading https://git-scm.com/book/en/v2/Git-Tools-Submodules

like image 23
Michael liv. Avatar answered Oct 24 '22 23:10

Michael liv.