Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

What does language_level in setup.py for cython do?

Tags:

cython

If I set language_level=2 in ext_modules = cythonize(extensions, language_level=2), what does that change? Is it just that the code I have written should be interpreted as Python2?

Is the end result exactly the same?

like image 756
The Unfun Cat Avatar asked Feb 27 '19 08:02

The Unfun Cat


People also ask

How does Cython work?

Cython works by producing a standard Python module. However, the behavior differs from standard Python in that the module code, originally written in Python, is translated into C. While the resulting code is fast, it makes many calls into the CPython interpreter and CPython standard libraries to perform actual work.

How do I import PYX files?

pyx. Complete documentation to build a setup file for the Cython. Open the terminal in the same directory you have the Cython file and compile your setup.py. After compiling the Cython, you can import it into the Python project.


1 Answers

Building a cython extension is a two-step proccess:

  1. creating the foo.c-file from foo.pyx file using PythonX+cython-module. X could be here 2.7, 3.7 or whatever version you prefer.
  2. creating the corresponding so-file (or pyd on Windows) with help of compiler and includes of PythonY and corresponding shared library. Here Y doesn't have to be X, but in most cases Y and X are the same.

The resulting extension can be used with PythonY (it doesn't play a role what X was).

However, there is still the question: In which Python-version was the original pyx-file written? If language_level is not set, current Cython-versions assume that the pyx-file was written in the version 2 (btw. this is not the case for IPython-%%cython-magic, where the version with which the file foo.c is cythonized).

This behavior will change in the future, this is the reason you see the somewhat irritating warning, if you build with cython>=0.29:

/Main.py:367: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: XXXXXX.pyx
tree = Parsing.p_module(s, pxd, full_module_name)

So you can explicitly set the language_level, so that your extension has the same behavior independent of the Cython-version with which it was cythonized.

For some examples of different behavior see, the following example.

Using language_level=3:

%%cython -3
print("I'm", "not a tuple")
print(5/4) 

results in

I'm not a tuple
1.25  

but using language_level=2:

%%cython -2
print("I'm", "not a tuple")
print(5/4) 

results in

("I'm", 'not a tuple')   # yet a tuple!
1                        # integer division in Python2!

Obviously the above are only two examples, there are much more differences (e.g. str & unicode stuff).


One of other notable differences is that Python3 disables implicit relative imports, that means inside of a package, we no longer cimport using implicit relative import

# works with language_level=2
cimport other_local_cymodule

but use explicit relative import

 # works with language_level=3,3str
 from . cimport other_local_cymodule

or absolute import

 # works with language_level=3,3str
 cimport package.other_local_cymodule

In general I would try to avoid mixing different language_level and Python-interpreter-version, as it can lead to counter-intuitive behavior.

For example in the following example mixing language_level=2 and Python3:

%%cython -2
def divide2(int a, int b):
    return a/b

def divide3(a, b):
    return a/b

>>> divide2(2,3), divide3(2,3)
# (0, 0.66666666) 

For the function divide2 Cython can ensure the "right" Python2-behavior, but how the division is performed depends on the behavior of int-object, which has the normal Python3-behavior.

like image 185
ead Avatar answered Oct 03 '22 17:10

ead