unsupported operand in pandas dataframe operation

Question

I have pandas DataFrame. I would like to get single value from a column based on a condition involving two another column. I am looking for the value from the column3 for which is the biggest distance in the column1 and 2.

I build the simple example which works:

d = pd.DataFrame({'c1':[.1,3,11.3],'c2':[3,6,.6],'c3':[8,.8,10.9]})
print'data d=\n%s\n' % d                                               
x = float(d.c3[abs(d.c1-d.c2)==max(abs(d.c1-d.c2))].values)
print 'the value of x= \n%s\n' % x

The output from this example is as I expect:

     c1   c2    c3
0   0.1  3.0   8.0
1   3.0  6.0   0.8
2  11.3  0.6  10.9

the value of x= 
10.9

I try to apply exactly the same logic to my original problem with large dataframe inside a class. The code is:

yInit = float(self.DenFrame.Depth[abs(self.DenFrame.Hper-self.DenFrame.Vper)==max(abs(self.DenFrame.Hper-self.DenFrame.Vper))].values)

but this code produce an error:

...
  File "C:\Python27\lib\site-packages\pandas-0.9.0-py2.7-win32.egg\pandas\core\series.py", line 73, in wrapper
return Series(na_op(self.values, other.values),
  File "C:\Python27\lib\site-packages\pandas-0.9.0-py2.7-win32.egg\pandas\core\series.py", line 59, in na_op
result[mask] = op(x[mask], y[mask])
TypeError: unsupported operand type(s) for -: 'str' and 'str'

I found in here that there could be a problem with type of the columns but Depth is type numpy.float64 Hper is type float Vper is type float so I understand how it can apply to my problem.

I don't know from this point what to do as I understand the same code works in one case but not in another and I cannot spot the problem.

Answer 1

You have some strings in your DenFrame.Hper and DenFrame.Vper.

You can see this by checking the dtype or type of each element:

In [11]: df.Hper.dtype
Out[11]: dtype('object')

Means that the numpy array could contain various types, we can see what these types are:

In [12]: DenFrame.Hper.map(type).unique()
Out[12]: [<type 'float'> <type 'str'>]

And you could inspect which entries are strings:

DenFrame[DenFrame.Hper.map(type) == str]

Perhaps it makes sense to only include those which are floats:

DenFrame_floats = DenFrame[(DenFrame.Hper.map(type) == float) & 
                           (DenFrame.Vper.map(type) == float)]

or you could (if it's possible) convert them to floats:

DenFrame.Hper = DenFrame.Hper.apply(float)