What does language_level in setup.py for cython do?

Question

If I set language_level=2 in ext_modules = cythonize(extensions, language_level=2), what does that change? Is it just that the code I have written should be interpreted as Python2?

Is the end result exactly the same?

Answer 1

Building a cython extension is a two-step proccess:

1. creating the foo.c-file from foo.pyx file using PythonX+cython-module. X could be here 2.7, 3.7 or whatever version you prefer.
2. creating the corresponding so-file (or pyd on Windows) with help of compiler and includes of PythonY and corresponding shared library. Here Y doesn't have to be X, but in most cases Y and X are the same.

The resulting extension can be used with PythonY (it doesn't play a role what X was).

However, there is still the question: In which Python-version was the original pyx-file written? If language_level is not set, current Cython-versions assume that the pyx-file was written in the version 2 (btw. this is not the case for IPython-%%cython-magic, where the version with which the file foo.c is cythonized).

This behavior will change in the future, this is the reason you see the somewhat irritating warning, if you build with cython>=0.29:

/Main.py:367: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: XXXXXX.pyx
tree = Parsing.p_module(s, pxd, full_module_name)

So you can explicitly set the language_level, so that your extension has the same behavior independent of the Cython-version with which it was cythonized.

For some examples of different behavior see, the following example.

Using language_level=3:

%%cython -3
print("I'm", "not a tuple")
print(5/4) 

results in

I'm not a tuple
1.25  

but using language_level=2:

%%cython -2
print("I'm", "not a tuple")
print(5/4) 

results in

("I'm", 'not a tuple')   # yet a tuple!
1                        # integer division in Python2!

Obviously the above are only two examples, there are much more differences (e.g. str & unicode stuff).

---

One of other notable differences is that Python3 disables implicit relative imports, that means inside of a package, we no longer cimport using implicit relative import

# works with language_level=2
cimport other_local_cymodule

but use explicit relative import

 # works with language_level=3,3str
 from . cimport other_local_cymodule

or absolute import

 # works with language_level=3,3str
 cimport package.other_local_cymodule

---

In general I would try to avoid mixing different language_level and Python-interpreter-version, as it can lead to counter-intuitive behavior.

For example in the following example mixing language_level=2 and Python3:

%%cython -2
def divide2(int a, int b):
    return a/b

def divide3(a, b):
    return a/b

>>> divide2(2,3), divide3(2,3)
# (0, 0.66666666) 

For the function divide2 Cython can ensure the "right" Python2-behavior, but how the division is performed depends on the behavior of int-object, which has the normal Python3-behavior.