Applying a function along a numpy array

Question

I've the following numpy ndarray.

[ -0.54761371  17.04850603   4.86054302] 

I want to apply this function to all elements of the array

def sigmoid(x):   return 1 / (1 + math.exp(-x))  probabilities = np.apply_along_axis(sigmoid, -1, scores) 

This is the error that I get.

TypeError: only length-1 arrays can be converted to Python scalars 

What am I doing wrong.

Answer 1

Function numpy.apply_along_axis is not good for this purpose. Try to use numpy.vectorize to vectorize your function: https://docs.scipy.org/doc/numpy/reference/generated/numpy.vectorize.html This function defines a vectorized function which takes a nested sequence of objects or numpy arrays as inputs and returns an single or tuple of numpy array as output.

import numpy as np import math  # custom function def sigmoid(x):   return 1 / (1 + math.exp(-x))  # define vectorized sigmoid sigmoid_v = np.vectorize(sigmoid)  # test scores = np.array([ -0.54761371,  17.04850603,   4.86054302]) print sigmoid_v(scores) 

Output: [ 0.36641822 0.99999996 0.99231327]

Performance test which shows that the scipy.special.expit is the best solution to calculate logistic function and vectorized variant comes to the worst:

import numpy as np import math import timeit  def sigmoid_(x):   return 1 / (1 + math.exp(-x)) sigmoidv = np.vectorize(sigmoid_)  def sigmoid(x):    return 1 / (1 + np.exp(x))  print timeit.timeit("sigmoidv(scores)", "from __main__ import sigmoidv, np; scores = np.random.randn(100)", number=25),\ timeit.timeit("sigmoid(scores)", "from __main__ import sigmoid, np; scores = np.random.randn(100)",  number=25),\ timeit.timeit("expit(scores)", "from scipy.special import expit; import numpy as np;   scores = np.random.randn(100)",  number=25)  print timeit.timeit("sigmoidv(scores)", "from __main__ import sigmoidv, np; scores = np.random.randn(1000)", number=25),\ timeit.timeit("sigmoid(scores)", "from __main__ import sigmoid, np; scores = np.random.randn(1000)",  number=25),\ timeit.timeit("expit(scores)", "from scipy.special import expit; import numpy as np;   scores = np.random.randn(1000)",  number=25)  print timeit.timeit("sigmoidv(scores)", "from __main__ import sigmoidv, np; scores = np.random.randn(10000)", number=25),\ timeit.timeit("sigmoid(scores)", "from __main__ import sigmoid, np; scores = np.random.randn(10000)",  number=25),\ timeit.timeit("expit(scores)", "from scipy.special import expit; import numpy as np;   scores = np.random.randn(10000)",  number=25) 

Results:

size        vectorized      numpy                 expit N=100:   0.00179314613342 0.000460863113403 0.000132083892822 N=1000:  0.0122890472412  0.00084114074707  0.000464916229248 N=10000: 0.109477043152   0.00530695915222  0.00424313545227 

Answer 2

Use np.exp and that will work on numpy arrays in a vectorized fashion:

>>> def sigmoid(x): ...     return 1 / (1 + np.exp(-x)) ... >>> sigmoid(scores) array([  6.33581776e-01,   3.94391811e-08,   7.68673281e-03]) >>> 

You will likely not get any faster than this. Consider:

>>> def sigmoid(x): ...     return 1 / (1 + np.exp(-x)) ... 

And:

>>> def sigmoidv(x): ...   return 1 / (1 + math.exp(-x)) ... >>> vsigmoid = np.vectorize(sigmoidv) 

Now, to compare the timings. With a small (size 100) array:

>>> t = timeit.timeit("vsigmoid(arr)", "from __main__ import vsigmoid, np; arr = np.random.randn(100)", number=100) >>> t 0.006894525984534994 >>> t = timeit.timeit("sigmoid(arr)", "from __main__ import sigmoid, np; arr = np.random.randn(100)", number=100) >>> t 0.0007238480029627681 

So, still an order-of-magnitude difference with small arrays. This performance differences stays relatively constant, with a 10,000 size array:

>>> t = timeit.timeit("vsigmoid(arr)", "from __main__ import vsigmoid, np; arr = np.random.randn(10000)", number=100) >>> t 0.3823414359940216 >>> t = timeit.timeit("sigmoid(arr)", "from __main__ import sigmoid, np; arr = np.random.randn(10000)", number=100) >>> t 0.011259705002885312 

And finally with a size 100,000 array:

>>> t = timeit.timeit("vsigmoid(arr)", "from __main__ import vsigmoid, np; arr = np.random.randn(100000)", number=100) >>> t 3.7680041620042175 >>> t = timeit.timeit("sigmoid(arr)", "from __main__ import sigmoid, np; arr = np.random.randn(100000)", number=100) >>> t 0.09544878199812956