How to get second __init__.py execute for same namespace

Question

Here is my directory structure

.
|-- path1
|   `-- mynms
|       |-- __init__.py
|       `-- app1
|           |-- __init__.py
|           `-- foo.py
|-- path2
|   `-- mynms
|       |-- __init__.py
|       `-- app2
|           |-- __init__.py
|           `-- bar.py
`-- user.py

File contents:

$ cat user.py
#!/usr/bin/python

import sys
sys.path.append('path1')
sys.path.append('path2')

from mynms.app2.foo import foo
from mynms.app2.bar import bar

foo()
bar()

$ cat path1/mynms/__init__.py;echo ==============;cat path2/mynms/__init__.py
from pkgutil import extend_path
__path__ = extend_path(__path__, __name__)

print "I am path1/mynms/__init__.py"
==============
from pkgutil import extend_path
__path__ = extend_path(__path__, __name__)

print "I am path2/mynms/__init__.py"

$ cat path1/mynms/app1/foo.py; echo ============; cat path2/mynms/app2/bar.py
def foo():
  print "foo!"
============
def bar():
  print "bar!"

Question: When I run user.py, I get only the output of path1/__init__.py but not for path2. Is there some way to fix that?

$ ./user.py
I am path1/mynms/__init__.py        -----> Why is 'I am path2/mynms/__init__.py not printed?
foo!
bar!

Answer 1

When you write import mynms.app2.foo or from mynms.app2.foo import foo, Python does this:

• import mynms by looking for the file mynms/__init__.py, which it finds in path1
• import mynms.app1 by looking for a file mynms/app1/__init__.py (or mynms/app1.py)
• import mynms.app1.foo by looking for mynms/app1/foo.py (or mynms/app1/foo/__init__.py)

On the next import statement, import mynms.app2.bar, Python does:

• import mynms - it's already imported so there's nothing to do. (You can check sys.modules['mynms'] to see whether it is already imported. If it isn't, it will raise KeyError.)
• import mynms.app2 - again, this is already imported.
• import mynms.app2.bar by reading the file mynms/app2/bar.py, which is in path2

There's no (sensible) way for Python to import the mynms module twice from two different files.

If you have initialisation code you will need to put them in the modules mynms.app1 and mynms.app2 so they have different names. i.e. the files mynms/app1/__init__.py and mynms/app2/__init__.py.

Answer 2

As Martijn Pieters has already mentioned in comments, Python 2 does not support packages distributed as a collection of packages. But you can emulate such packages if you use setuptools (most probably you do).

The only problem is that the packages should have correct metainformation, so that setuptools could join several subpackages in a virtual single package. The information is usually passed to setup() function in setup.py and lives in your site-packages directory after the installation. That means that those who distribute the packages should anyway make something to properly prepare them, and you can hardly just put them in PATH to make them work.

Here is the link to the corresponding documentation. Be ready to spend some time, because PEAK documentation is not always enough for a quick start.