Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Can I cleanse a numpy array without a loop?

Tags:

python

numpy

Perhaps not such a big deal, but it breaks my heart to follow this:

deltas = data[1:] - data[:-1]

with this:

for i in range(len(deltas)):
        if deltas[i] < 0: deltas[i] = 0
        if deltas[i] > 100: deltas[i] = 0

For this particular example...is there a better way to do the cleansing part?

Question part two: What if the cleansing rules are more complicated, or less complicated than this example. For example, we might just want to change all negative numbers to zero. Or, we might be doing a more complicated mapping.

like image 605
Pete Avatar asked Dec 02 '10 19:12

Pete


People also ask

How do I clean my NumPy array?

Using the NumPy function np. delete() , you can delete any row and column from the NumPy array ndarray . Specify the axis (dimension) and position (row number, column number, etc.). It is also possible to select multiple rows and columns using a slice or a list.

Why NumPy array operations are faster than looping?

NumPy is fast because it can do all its calculations without calling back into Python. Since this function involves looping in Python, we lose all the performance benefits of using NumPy. For a 10,000,000-entry NumPy array, this functions takes 2.5 seconds to run on my computer.

Does NumPy use for loops?

Numpy for loop is used for iterating through numpy arrays of different dimensions, which is created using the python numpy library and using the for loop, multiple operations can be done going through each element in the array by one.

Does NumPy do lazy evaluation?

WeldNumpy is a Weld-enabled library that provides a subclass of NumPy's ndarray module, called weldarray, which supports automatic parallelization, lazy evaluation, and various other optimizations for data science workloads.


2 Answers

import numpy as np
deltas=np.diff(data)
deltas[deltas<0]=0
deltas[deltas>100]=0

Also possible, and a bit quicker is

deltas[(deltas<0) | (deltas>100)]=0
like image 132
unutbu Avatar answered Sep 18 '22 13:09

unutbu


Try using numpy.vectorize to apply a function to each element of the numpy array.

like image 34
Karl Knechtel Avatar answered Sep 20 '22 13:09

Karl Knechtel