What are the best practices for sharing a module among subpackages?

Question

Suppose you're creating a python package with four subpackages, and there is a module generic_stuff.py which is required by exactly two of them. To make it concrete, perhaps generic_stuff.py contains some custom functions for reading from and writing to a database and subpackages 1 and 2 contain code for web servers which are to be deployed separately.

The question is:

What is the right way to make generic_stuff.py available to subpackages 1 and 2?

I can think of three ways to do it, but all of them feel like hacks:

1. Make generic_stuff.py part of its own package and install it. This is unsatisfactory because it pollutes the namespace of the whole system (not to mention subpackages 3 and 4).
2. Add the parent directory (containing generic_stuff.py and the four subpackages) to the system path before importing in subpackages 1 and 2. This is unsatisfactory because it adds everything in the parent directory to the namespace of subpackages 1 and 2. This could be avoided by putting generic_stuff.py in its own directory and adding that to the system path, but even then this approach doesn't scale very well if there are multiple generic modules which are being used by multiple modules in subpackages 1 and 2 (imagine trying to refactor that project).
3. Maintain two copies of generic_stuff.py: one in subpackage 1 and subpackage 2. Do I need to explain why this is a bad idea?

I suppose what I have in mind is some sort of configuration, perhaps which would go in the parent directory's __init__.py or something, that would make module X available to subpackage Y. Does anything like this exist?

Answer 1

Maybe I'm something, but it seems like you're overthinking the problem. Simply put generic_stuff.py at the highest level of your package and import it in both sub1 and sub2. You can use absolute or relative imports, no name pollution no messing with sys.path. For example:

Package structure:

package
|-- generic_stuff.py
|-- __init__.py
|-- sub1
|   |-- __init__.py
|   `-- sub1_module.py
|-- sub2
|   `-- __init__.py
|-- sub3
|   `-- __init__.py
`-- sub4
    `-- __init__.py

In all modules that need generic_stuff do something like:

# sub1_module.py
# import using a relative reference
import ..generic_stuff as gs
gs.some_tool()
# or an absolute reference
from package import generic_stuff as gs
gs.some_tool()