Python project and package directories layout

Question

I created a project in python, and I'm curious about how packages work in python.

Here is my directory layout:

top-level dir
  \ tests
    __init__.py 
  \ examples
    __init__.py
    example.py
module.py

How would I go about including module.py in my example.py module. I know I could set PYTHONPATH to the top-level directory, but that doesn't seem like a good solution. This is how pydev gets around this problem, but I'd like a solution that didn't require updating environment variables.

I could put something at the top of the example.py to update sys.path like this:

from os import path
import sys

sys.path.append( path.dirname(path.abspath(path.dirname(__file__))) )

I don't think this is an appropriate solution either.

I feel like I'm missing some basic part of python packages. I'm testing this on python 2.6. If any further clarification is needed, please let me know.

Answer 1

Python packages are very simple: a package is any directory under any entry in sys.path that has an __init__.py file. However, a module is only considered to be IN a package if it is imported via a relative import such as import package.module or from package import module. Note that this means that in general, someone must set up sys.path to contain the directories above any package you want to be importable, whether via PYTHONPATH or otherwise.

The primary wrinkle is that main modules (those run directly from the command line) are always __main__ no matter their location. They therefore have to do absolute imports, and either rely on PYTHONPATH being set up, or munge sys.path themselves.

In your case, I'd recommend either having a small Python script that runs your examples after setting up the correct path. Say you put it in the top level directory:

#!/usr/bin/env python
import sys
import os.path

sys.path.append(os.path.dirname(__file__))
example = __import__("examples", globals(), locals(), sys.argv[1])

Then your example can do "import module".

Alternately, if module.py is meant to also be in a package, add the PARENT of your "top level directory" to sys.path and then use the from .. import module syntax in your example modules. Also, change the first parameter to the __import__ in the wrapper to "tldname.examples" where tldname is the name of your top-level directory.

If you don't want to rely on an "example runner" module, each example needs sys.path boilerplate, or you need a PYTHONPATH setting. Sorry.