how to document cython function on readthedocs

Question

On ReadTheDocs I am not allowed to compile cython extensions, is it possible to configure sphinx in order to extract the docstrings from the cython files without actually compiling them?

thanks!

Answer 1

I faced the same problem and found that it is possible now to compile Cython extensions on readthedocs.

Short answer: Cython modules can be compiled using the virtualenv feature provided by readthedocs.

For a slightly longer answer and an example project see below.

What is the problem?

As I have understood it, sphinx imports all modules of the project that shall be documented and then extracts the docstrings in python. This fails for Cython modules, as they cannot be directly imported and have to be compiled first. Compiling the modules does not work out of the box on readthedocs, but they are providing a tool to realise this.

How to solve this.

When installing the project in a virtualenv, the Cython modules will be build (into .so files) and can then be imported. This might require some external modules though (numpy for the example below and, of course, Cython). These can be specified in a pip requirements file (requirements.txt) that has to be in your repository.

1. Enable the option install your project inside a virtualenv under Admin -> Advanced Settings on readthedocs
2. Enter the path to your requirements.txt relative to the project root (docs/requirements.txt in the example below)
3. (If necessary change the python interpreter version)

Now your project will be installed (using python setup.py install) each time the documentation is build. The output of the setup script can be seen under Setup Output if you click on the respective build in the Builds tab on readthedocs. This is where compile time errors might show up. Mind that compiling your project might take some time.

Example project

A Python package consisting of several Cython modules, each of which has Google-style docstrings.

my_project/
    setup.py
    my_package/
        __init__.py   # imports Cython modules
        cython_module1.pyx
        cython_module2.pyx
        ...
    docs/
        requirements.txt
        Makefile
        source/
            conf.py
            index.rst
            ... #  more documentation

requirements.txt

cython>=0.20
numpy>=1.9

Caveat(s)

While trying this on my project I ran into the problem that my Cython modules could not be imported. The error message of sphinx read like:

home/docs/checkouts/readthedocs.org/user_builds/... :4: WARNING: autodoc: failed to import module 'cython_module1';...
File "/home/docs/checkouts/readthedocs.org/user_builds/.../__init__.py", ...
from .cython_module1 import CythonClass

This happened, because I used to build my documentation locally and had added a line like

...
sys.path.insert(0, os.path.abspath('../../')) # path to my_package
...

to my conf.py as suggested here. This had solved my problem when building locally (where I compiled my project using python setup.py build_ext --inplace), but when installing in the virtualenv this points to the wrong version of my_package (namely the sources, not the installed package). There sphinx can not find any .so files to import.

To solve this, just remove the line completely.

I hope this helps.