Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How do I remove rows from a numpy array based on multiple conditions?

I have a file with three columns and thousands of rows. I want to remove those rows, whose items in the first column are in a certain range. For example, if the data in my file is as follows:

18  6.215   0.025
19  6.203   0.025
20  6.200   0.025
21  6.205   0.025
22  6.201   0.026
23  6.197   0.026
24  6.188   0.024
25  6.187   0.023
26  6.189   0.021
27  6.188   0.020
28  6.192   0.019
29  6.185   0.020
30  6.189   0.019
31  6.191   0.018
32  6.188   0.019
33  6.187   0.019
34  6.194   0.021
35  6.192   0.024
36  6.193   0.024
37  6.187   0.026
38  6.184   0.026
39  6.183   0.027
40  6.189   0.027

I want to remove those rows, whose first item is between 20 and 25 or between 30 and 35. The expected output is thus:

18  6.215   0.025
19  6.203   0.025
26  6.189   0.021
27  6.188   0.020
28  6.192   0.019
29  6.185   0.020
36  6.193   0.024
37  6.187   0.026
38  6.184   0.026
39  6.183   0.027
40  6.189   0.027

How could I do this?

like image 244
Jun Avatar asked Aug 19 '14 09:08

Jun


People also ask

How do I delete rows in NumPy array based on condition?

np. delete(ndarray, index, axis): Delete items of rows or columns from the NumPy array based on given index conditions and axis specified, the parameter ndarray is the array on which the manipulation will happen, the index is the particular rows based on conditions to be deleted, axis=0 for removing rows in our case.

How do I delete multiple rows in NumPy?

delete() – The numpy. delete() is a function in Python which returns a new array with the deletion of sub-arrays along with the mentioned axis. By keeping the value of the axis as zero, there are two possible ways to delete multiple rows using numphy. delete().

How do I remove multiple elements from a NumPy array?

The correct way to remove multiple elements is to: Add the indices to a sequence, such as a list. Call the numpy. delete() function on the array with the given index sequence.


Video Answer


1 Answers

If you want to keep using numpy, the solution isn't hard.

data = data[np.logical_not(np.logical_and(data[:,0] > 20, data[:,0] < 25))]
data = data[np.logical_not(np.logical_and(data[:,0] > 30, data[:,0] < 35))]

Or if you want to combine it all into one statement,

data = data[
    np.logical_not(np.logical_or(
        np.logical_and(data[:,0] > 20, data[:,0] < 25),
        np.logical_and(data[:,0] > 30, data[:,0] < 35)
    ))
]

To explain, conditional statements like data[:,0] < 25 create boolean arrays that track, element-by-element, where the condition in an array is true or false. In this case, it tells you where the first column of data is less than 25.

You can also index numpy arrays with these boolean arrays. A statement like data[data[:,0] > 30] extracts all the rows where data[:,0] > 30 is true, or all the rows where the first element is greater than 30. This kind of conditional indexing is how you extract the rows (or columns, or elements) that you want.

Finally, we need logical tools to combine boolean arrays element-by-element. Regular and, or, and not statements don't work because they try to combine the boolean arrays together as a whole. Fortunately, numpy provides a set of these tools for use in the form of np.logical_and, np.logical_or, and np.logical_not. With these, we can combine our boolean arrays element-wise to find rows that satisfy more complicated conditions.

like image 122
Roger Fan Avatar answered Sep 20 '22 13:09

Roger Fan