Using numba with time.clock() and timeit

Question

I am trying to test numba and numpy on a very simple example and check the efficiency. However,

1. Using time.clock and timeit I have very different results in case of numba.

2. In case of time.clock it shows that using numba with numpy makes the function slower.

3. Does it make sense to use vectorize() for NbNpFunc?

Here is my code. Thanks for the help.

import numpy as np
from numba import jit
import time
import timeit
#import math
#import matplotlib.pyplot as plt
#import matplotlib.animation as animation

N = 10000

def PyFunc(N):
    r = list(range(0,N)) 
    for i in range(0,len(r)):
            r[i] += r[i]*r[i]
    return(r)

def NpFunc(N):
    r = np.arange(0,N)
    r += r*r
    return(r)

@jit
def NbFunc(N):
    r = list(range(0,N))
    for i in range(0,len(r)):
        r[i] += r[i]*r[i]
    return(r)

@jit
def NbNpFunc(N):
    r = np.arange(0,N) 
    r += r*r
    return(r)

print("\nUsing time.clock()")
start_time = time.clock()
res1 = PyFunc(N)
print("PyFunc --- %s seconds ---"  %(time.clock() - start_time))    
start_time = time.clock()
res2 = NpFunc(N)
print("NumPyFunc --- %s seconds ---" % (time.clock() - start_time))    
start_time = time.clock()
res3 = NbFunc(N)
print("NumbaFunc --- %s seconds ---" % (time.clock() - start_time))    
start_time = time.clock()
res4 = NbNpFunc(N)
print("NumbaNpFunc --- %s seconds ---" % (time.clock() - start_time))    

print("\nUsing timeit")
t = timeit.Timer(lambda: PyFunc(N))
print ("PyFunc --- %s seconds ---" %t.timeit(number=1))
t = timeit.Timer(lambda: NpFunc(N))
print ("NumPyFunc --- %s seconds ---" %t.timeit(number=1))
t = timeit.Timer(lambda: NbFunc(N))
print ("NumbaFunc --- %s seconds ---" %t.timeit(number=1))
t = timeit.Timer(lambda: NbNpFunc(N))
print ("NumbaNpFunc --- %s seconds ---" %t.timeit(number=1))

Answer 1

Your benchmark has several problems

• You have to call each function once before measuring performance. Otherwise you are measuring the compilation overhead
• Timers are not that precise, you have to run each function several times
• Even then results may differ, because you have a very short running function, in this case function call overhead may be a problem
• You are not really measuring the calculation speed. Memory allocation is the most costly part of your code. The timings for memory allocation are heavily influenced by the garbage collector and other things the kernel has to do at benchmark time

Proposals for better benchmarking

• Allocate memory bevorehand

• Create a longer running code (increase N)

• Do more calculation in your code (sin,cos,sqrt), in case of benchmarking simple multiplications you are benchmarking RAM-speed and cache-speed

• Take a look at nb.njit(fastmath=True)

• Make sure to have the newest Numba version and to have installed Intel SVML