Using numpy ufuncs vs built-in operators

Question

I'm curious about the benefits and tradeoffs of using numpy ufuncs vs. the built-in operators vs. the 'function' versions of the built-in operators.

I'm curious about all ufuncs. Maybe there are times when some are more useful than others. However, I'll use < for my examples just for simplicity.

There are several ways to 'filter' a numpy array by a single number to get a boolean array. Each form gives the same results, but is there a preferred time/place to use one over the other? This example I'm comparing an array against a single number, so all 3 will work.

Consider all examples using the following array:

>>> x = numpy.arange(0, 10000)
>>> x
array([   0,    1,    2, ..., 9997, 9998, 9999])

'<' operator

>>> x < 5000
array([ True,  True,  True, ..., False, False, False], dtype=bool)
>>> %timeit x < 5000
100000 loops, best of 3: 15.3 us per loop

operator.lt

>>> import operator
>>> operator.lt(x, 5000)
array([ True,  True,  True, ..., False, False, False], dtype=bool)
>>> %timeit operator.lt(x, 5000)
100000 loops, best of 3: 15.3 us per loop

numpy.less

>>> numpy.less(x, 5000)
array([ True,  True,  True, ..., False, False, False], dtype=bool)
>>> %timeit numpy.less(x, 5000)
100000 loops, best of 3: 15 us per loop

Note that all of them achieve pretty much the equivalent performance and exactly the same results. I'm guessing that all of these calls actually end up in the same function anyway since < and operator.lt both map to __lt__ on a numpy array, which is probably implemented using numpy.less or the equivalent?

So, which is more 'idiomatic' and 'preferred'?

Answer 1

Generally speaking, thinking of the "readability counts" mantra, the actual operator should always be your preferred choice. Using the operator versions has a place, when you can replace lambda a, b: a < b with the more compact operator.lt, but not much outside of that. And you really shouldn't be using explicit calls to the corresponding ufunc, unless you want to use the out parameter to store the calculated values directly in an existing array.

That said, if what you are worried is performance, you should do fair comparisons, because as you say, all your calls are eventually handled by numpy's less ufunc.

If your data is already in a numpy array, then you have already shown that they are all performing similarly, so go with the < operator for clarity.

What if your data is in a python object, say a list? Well, here are some timings for you to ponder:

In [13]: x = range(10**5)

In [19]: %timeit [j < 5000 for j in x]
100 loops, best of 3: 5.32 ms per loop

In [20]: %timeit np.less(x, 5000)
100 loops, best of 3: 11.3 ms per loop

In [21]: %timeit [operator.lt(j, 5000) for j in x]
100 loops, best of 3: 16.2 ms per loop

Not sure why operator.lt is so slow, but you clearly want to stay away from it. If you want to get a numpy array as output from a Python object input, then this will probably be the fastest:

In [22]: %timeit np.fromiter((j < 5000 for j in x), dtype=bool, count=10**5)
100 loops, best of 3: 7.91 ms per loop

Note that ufuncs operating on numpy arrays are much faster than any of the above:

In [24]: y = np.array(x)

In [25]: %timeit y < 5000
10000 loops, best of 3: 82.8 us per loop