Cannot cast array data from dtype('O') to dtype('float64')

Question

I am using scipy's curve_fit to fit a function to some data, and receive the following error;

Cannot cast array data from dtype('O') to dtype('float64') according to the rule 'safe'

which points me to this line in my code;

popt_r, pcov = curve_fit(
                    self.rightFunc, np.array(wavelength)[beg:end][edgeIndex+30:], 
                    np.dstack(transmitted[:,:,c][edgeIndex+30:])[0][0],
                    p0=[self.m_right, self.a_right])

rightFunc is defined as follows;

def rightFunc(self, x, m, const):

    return np.exp(-(m*x + const))

As I understand it, the 'O' type refers to a python object, but I can't see what is causing this error.

Complete Error:

Any ideas for what I should investigate to get to the bottom of this?

Answer 1

Just in case it could help someone else, I used numpy.array(wavelength,dtype='float64') to force the conversion of objects in the list to numpy's float64. Works well for me.

Answer 2

Typically these scipy functions require parameters like:

curvefit( function, initial_values, (aux_values,), ...)

where the tuple of aux_values is passed through to your function along with the current value of the main variable.

Is the dstack expression this aux_values? Or a concatenation of several. It may need to be wrapped in a tuple.

(np.dstack(transmitted[:,:,c][edgeIndex+30:])[0][0],)

We may need to know exactly where this error arises, not just which line of your code does it. We need to know what value is being converted. Where is there an array with dtype object?

Answer 3

Just to clarify, I had the same problem, did not see the right answers in the comments, before solving it on my own. So I just repeat them here:

I have resolved the issue. I was passing an array with one element to the p0 list, rather than the element itself. Thank you for your help – Jacobadtr Sep 12 at 17:51

An O dtype often results when constructing an array from a list of sublists that differ in size. If np.array(...) can't make a clean n-d array of numbers, it resorts to making an array of objects. – hpaulj Sep 12 at 17:15

That is, make sure that the tuple of parameters you pass to curve_fit can be properly casted to an numpy array