Buildout with part build with Cython

Question

I'm facing problem with cython in buildout.

One of the part is a module build with cython from a .c file and a .pyx file.

I've already try many solutions :

• Sean Gillies Blog / 814 / Adding pyproj to a buildout
• gefira blog » Getting rid of ‘No Cython, trying Pyrex…’ in zc.buildout

But all ends with error :

ImportError: No module named Cython.Distutils

This append when buildout load the setup.py of this cython module.

Note that the setup.py is functional when called from its own directory and with an interpreter in buildout bin directory.

Thanks for your help.

Answer 1

The articles you linked tell you how to install Cython from a buildout, but to do that you need write access to the site-packages folder. There are 3 ways you should be able to run those buildouts:

1. Run it as root and install Cython into the system python's site-packages. Usually you want to avoid doing this.

2. Compile your own Python. This may be the only option if you want to use a version of Python that is not already on the system. There are buildout configurations that will let you build any version of Python from inside the buildout.

3. Use virtualenv. This will create (in the buildout folder) a complete virtual environment for Python including your own site-packages folder. This is usually the best way to run a buildout that can use the system Python (or any other Python that you have already installed system wide).

I suggest you make using virtualenv part of the installation instructions for your software.

Answer 2

I found today this recipe https://pypi.python.org/pypi/mr.cython/1.0 which solve the problem.

This solve the problem by installing cython with an extention recipe so it is available when buildout run setup.py develop

The problem was to build a cython module without having cython installed as system level.

Answer 3

There is no way to do this as a single step, but it's easy to do as a two step process.

Use:

python bootstrap.py
./bin/buildout install cython
./bin/cpy bootstrap.py
./bin/cpy ./bin/buildout

The reason this is possible is because buildout supports an obscure option 'install' which no one ever talks about, but you can use it, like this:

[buildout]
parts = deps py   # <---- Notice we don't depend on cython here
eggs =
  whatever
  kdist
  nark
  kivy # <--- But we do have a module that requires cython to build
develop =
  .
  lib/nark
  lib/kivy-dist

[cython] # <---- By calling ./bin/buildout install cython we trigger this
recipe = zc.recipe.egg:script
parts = cython-py
interpreter = cpy # <--- Which makes our dummy ./bin/cpy 
eggs =
  cython
  pyinstaller

[deps]
recipe = zc.recipe.egg:eggs
eggs = ${buildout:eggs}

[py]
recipe = zc.recipe.egg:script
interpreter = py
eggs = ${buildout:eggs}

The cute thing about this approach is that running buildout a second time clears out the bin directory so at the end of the day, you're left with a bin directory that looks like this:

$ ls bin/
buildout garden   py

No leftover packages that may or may not hang around in your virtualenv and screw things up later. That why we're using buildout in the first place right?

...of course, if you want cython to hang around, juts stick it into the dependencies at the top as well.