Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

numpy dot product with missing values

How do you do a numpy dot product where the two vectors might have missing values? This seems to require many additional steps, is there an easier way to do this?:

v1 = np.array([1,4,2,np.nan,3])
v2 = np.array([np.nan,np.nan,2,4,1])
np.where(np.isnan(v1),0,v1).dot(np.where(np.isnan(v2),0,v2))
like image 722
Allen Wang Avatar asked Jun 20 '17 17:06

Allen Wang


People also ask

How do I find missing values in Numpy?

Droping the missing values or nan values can be done by using the function "numpy. isnan()" it will give us the indexes which are having nan values and when combined with other function which is "numpy. logical_not()" where the boolean values will be reversed.

How do you make a dot product without Numpy?

Python dot product without NumPy If we don't have a NumPy package then we can define 2 vectors a and b. Then use zip function which accepts two equal-length vectors and merges them into pairs. Multiply the values in each pair and add the product of each multiplication to get the dot product.

How does dot product work in Numpy?

Numpy with PythonThis function returns the dot product of two arrays. For 2-D vectors, it is the equivalent to matrix multiplication. For 1-D arrays, it is the inner product of the vectors. For N-dimensional arrays, it is a sum product over the last axis of a and the second-last axis of b.


2 Answers

We can use np.nansum to sum up the values ignoring NaNs after element-wise multiplication -

np.nansum(v1*v2)

Sample run -

In [109]: v1
Out[109]: array([  1.,   4.,   2.,  nan,   3.])

In [110]: v2
Out[110]: array([ nan,  nan,   2.,   4.,   1.])

In [111]: np.where(np.isnan(v1),0,v1).dot(np.where(np.isnan(v2),0,v2))
Out[111]: 7.0

In [115]: v1*v2
Out[115]: array([ nan,  nan,   4.,  nan,   3.])

In [116]: np.nansum(v1*v2)
Out[116]: 7.0
like image 155
Divakar Avatar answered Sep 21 '22 01:09

Divakar


Another solution is to use masked arrays:

v1 = np.array([1,4,2,np.nan,3])
v2 = np.array([np.nan,np.nan,2,4,1])

v1_m = numpy.ma.array(v1, mask=numpy.isnan(v1))
v2_m = numpy.ma.array(v2, mask=numpy.isnan(v2))

numpy.ma.dot(v1_m, v2_m)
like image 23
dllahr Avatar answered Sep 19 '22 01:09

dllahr