Using git submodules with python

Question

I've read a lot of blog posts and questions on this site about the usage of git submodules and still have no idea how to better use them with python.

I mean, what is the easier way to manage dependencies if I have such a package:

├── mypkg
│   └── __init__.py
├── setup.py
└── submodules
    ├── subm1
    └── subm2

Then, what if I need to use "mypkg" as a submodule for "top_level_pkg":

├── setup.py
├── submodules
│   └── mypkg
└── top_level_package
    └── __init__.py

, I want to run pip install . and have all resolved correctly (have each submodule installed to the VENV in correct order).

What I've tried:

• Install each submodule using "pip" running in a subprocess. But it seems to be a hacky way and hard to manage (Unexpected installation of GIT submodule)
• Use "install_requires" with "setuptools.find_packages()" but without success
• Use requirements.txt file for each submodule, but I can't find a way how to automate it so "pip" could automatically install all requirements for all submodules.

Ideally, I imagine a separate setup.py file for each submodule with install_requires=['submodules/subm1', 'submodules/submn'], but setuptools does not support it.

Answer 1

I'm not saying it's impossible, but very hard and very tricky. A safer way is to turn each submodule into an installable Python module (with it's own setup.py) and install the submodules from Git.

This link describes how to install packages from Git with setup.py: https://stackoverflow.com/a/32689886/2952185

Answer 2

Thankfully to Gijs Wobben and sinoroc I came up with solution that works for my case:

install_requires=['subm1 @ file://localhost/<CURENT_DIR>/path/to/subm1']