Sphinx and relative imports in Python 3.*

Question

I have a directory with a python package as follows:

--docs/index.rst
--docs/...
--app/__init__.py
--app/foo.py

and I'm using sphinx with autodocs for documenting the app (in python 3.3).

Now, in the conf.py (inside docs/), I have

sys.path.insert(0, os.path.abspath('../app'))

I cd into docs/, run

make html

which gives me

SystemError: Parent module '' not loaded, cannot perform relative import

to all the modules that have a

from .foo import Bar

I have a clean virtualenv installation of Sphinx using

pip install Sphinx

after I created the (clean) environment for python 3.3.

What am I missing?

I was moving the project from python 2.* to python 3.* when this happened. All the project is working, but this...

Answer 1

Your app directory is a package. A package is a directory with __init.py__ and other files inside it.

If you put a package directory on your sys.path, all kinds of things go wrong.

Let's take an example:

root/
    app/
        app/__init__.py
        app/spam.py
        app/eggs.py

---

If you have root on your sys.path (because it's your current working directory, or because you do it explicitly, or because you've installed things correctly to your site-packages), then app is a package, app.spam is a module, and, within app.eggs, .spam is that module. So, everything works.

If you have app on your sys.path, then app is not a package, spam is a module, and, within eggs, .spam isn't anything. So, you can't use relative imports.

If you have both on your sys.path, then app is a package, spam and app.spam are both different modules (with the same contents, executed twice), and within app.eggs, .spam is a module, but within eggs, .spam isn't anything. This will cause you no end of problems.

---

So, most likely, the fix you want is this:

sys.path.insert(0, os.path.abspath('..'))

If there are other packages, or directories full of Python code that aren't packages, in .. that you don't want to autodoc (e.g., a tests directory with tests/test_spam.py), then you will need to restructure your directories to put app into some directory that doesn't have any other Python code in it, like this:

root/
    src/
        app/
    tests/
    doc/

---

Alternatively, if you didn't want app to be a package, but rather to be a sys.path root directory, then kill the __init__.py, and leave app directly in sys.path. But in that case, you can't use intra-package relative imports; all of the modules in app are top-level modules, and have to be imported as such.

---

The Packages section of the tutorial (and the rest of the chapter above it) explains some of this, but there's probably better introductory documentation out there.

For full details, in 3.3+, The import system has everything, nicely organized; for older versions, the reference docs are muddy, incomplete, and scattered; you have to start at The import statement, and then read The Knights Who Say Neeeow ... Wum ... Ping! (which is basically a PEP but 1.5 didn't have PEPs yet), and possibly even the ni documentation, if you can find it, plus various PEPs and minor change log entries that explain how things have changed between 1.5 and 2.7 or 3.2 or whatever.