Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Create a mask both for nan and inf values in an array

Tags:

python

nan

numpy

I have to remove both nan and inf values from two arrays. I found this post useful https://stackoverflow.com/a/48591908/7541421 for removing nan. Is there any similar solution when I can create a mask to remove both nan and inf values?

The example below is just illustrative, I have arrays of large dimensions (400 elements)

import numpy as np
from numpy import nan, inf

a = np.asarray([0.5, 6.2, np.nan, 4.5, np.inf])
b = np.asarray([np.inf, np.inf, 0.3, np.nan, 0.5])

bad = ~np.logical_or(np.isnan(a), np.isnan(b))

X = np.compress(bad, a)  
Y = np.compress(bad, b) 

BIAS = np.nanmean(X - Y)
RMSE = np.sqrt(np.nanmean((X - Y)**2))
CORR = np.corrcoef(X, Y)

I need this in order to get both the statistics and plots correctly

like image 480
PEBKAC Avatar asked Jun 15 '18 13:06

PEBKAC


2 Answers

You can use np.isfinite(). It will return a boolean mask with True wherever the values are neither infinite nor NAN.

You can get the finite values this way:

a = np.asarray(a)
a = a[np.isfinite(a)]

Or for both arrays together:

mask = np.isfinite(a) | np.isfinite(b)
a = a[mask]
b = b[mask]
like image 193
John Zwinck Avatar answered Sep 17 '22 22:09

John Zwinck


np.isfinite

Test element-wise for finiteness (not infinity or not Not a Number).

like image 40
YSelf Avatar answered Sep 19 '22 22:09

YSelf