cython and numpy: 'cfunc.to_py:65:25: 'ndarray' is not a type identifier'

Question

I have a cdef function which takes two numpy.ndarrays as argument (1). It gives me the an 'cfunc.to_py:65:25: 'ndarray' is not a type identifier' error. when I replace the cdef with a def (python) function (2) or a cpdef (3), the error disappears. Is this a bug? How can I solve this? code is below.

(1) cdef float F(np.ndarray xx, np.ndarray y_non_labor, float wage, float gamma, float one_by_gamma, float l, float one_minus_l, int lenx):

(2) def F(np.ndarray xx, np.ndarray y_non_labor, float wage, float gamma, float one_by_gamma, float l, float one_minus_l, int lenx):

(3) cpdef float F(np.ndarray xx, np.ndarray y_non_labor, float wage, float gamma, float one_by_gamma, float l, float one_minus_l, int lenx):

# cython: profile=True
# cython: boundscheck=False
# cython: wraparound=False
cimport cython
from libc.stdlib cimport malloc
from scipy.optimize import minimize
cimport numpy as np
import numpy as np


# due to instabilities associated with "constraint optimization" with bounds
# we will transform the problem so that that it becomes an unconstraint optimization problem without bounds

def transform(x):
    x1 = x * x / (1 + x * x)
    s =  1 - sum(x1)
    return x1, s

# on the next cython update try cdef float F(..)
@cython.cdivision(True)
cdef float F(np.ndarray xx, np.ndarray y_non_labor, float wage, float gamma, float one_by_gamma, float l, float one_minus_l, int lenx):
    cdef float s
    cdef float *z_non_labor = <float *>malloc(lenx * sizeof(float))
    cdef float z_labor
    cdef float labor
    cdef float value
    cdef float summation
    cdef float non_labor
    cdef float x
    cdef float ynl

    for i from 0 <= i < lenx:
        x = xx[i]
        z_non_labor[i] = x * x / (1 + x * x)

    s = 0
    for i from 0 <= i < lenx:
        s += z_non_labor[i]

    z_labor = 1 - s

    summation = 0
    for i from 0 <= i < lenx:
        ynl = y_non_labor[i]
        summation += (z_non_labor[i] / ynl) ** gamma

    non_labor = summation ** one_by_gamma
    labor = z_labor / wage
    # labor and non-labor inputs together
    value = (labor ** l) * (non_labor ** one_minus_l)
    return - value

def optimization(np.ndarray seed_weights,
                 np.ndarray input_prices,
                 float wage,
                 float gamma,
                 float one_by_gamma,
                 float l,
                 float one_minus_l):

    cdef int lenx

    lenx = len(seed_weights)

    args=(input_prices,
          wage,
          gamma,
          one_by_gamma,
          l,
          one_minus_l,
          lenx)

    return minimize(F, seed_weights, args=args, method='Nelder-Mead')

Answer 1

I think it's a fairly obscure bug. Explanation follows. Work-round is at the end of the answer.

On Cython 0.21 I get a different error

filename.pyx:73:21: Cannot convert 'float (ndarray, ndarray, float,
float, float, float, float, int)' to Python object.

This makes sense to me because minimise takes a callable Python object as its first answer, while cdef generates a C function only. [cpdef generates both and calls the C function where it can]

On Cython 0.22 and 0.23 I get the error you describe. The full error is

@cname("__Pyx_CFunc_float____ndarray____ndarray____float____float____float____float____float____int___to_py")
cdef object __Pyx_CFunc_float____ndarray____ndarray____float____float____float____float____float____int___to_py(float (*f)(ndarray, ndarray, float, float, float, float, float, int) except *):
    def wrap(ndarray xx, ndarray y_non_labor, float wage, float gamma, float one_by_gamma, float l, float one_minus_l, int lenx):
                        ^
------------------------------------------------------------

cfunc.to_py:30:25: 'ndarray' is not a type identifier

You'll note that this is for a function wrap which you don't define. wrap seems to be generated only because you're calling minimize, and it realises you need a Python object here, so helpfully makes a def function for you (if you comment out the call to minimize the error goes away). This appears to be an enhancement that was made around Cython 0.22.

Unfortunately, the code for wrap uses ndarray rather than np.ndarray as you've defined it (this has to be a bug!). The workround is to add the following line

from numpy cimport ndarray

(which makes ndarray on its own a valid type).

However, given minimize calls it through the Python-compatible wrapper by necessity, you might as well just use cpdef (which is also called through a Python-compatible wrapper, but one generated in a different way without the bug). There's pretty much no disadvantage to using cpdef - the function will still get called quickly as a pure C function where possible and work fine from Python too (albeit slightly slower).