.NET array is slower than list in IronPython?

Question

I did the following matrix multiplication benchmark in IronPython based on code here:

from System import Random
from System.Diagnostics import Stopwatch

def zero(m,n):
    # Create zero matrix
    new_matrix = [[0 for row in range(n)] for col in range(m)]
    return new_matrix

def rand(m,n):
    # Create random matrix
    rnd = Random(1)
    new_matrix = [[rnd.NextDouble() for row in range(n)] for col in range(m)]
    return new_matrix

def show(matrix):
    # Print out matrix
    for col in matrix:
        print col 

def mult(matrix1,matrix2):
    # Matrix multiplication
    if len(matrix1[0]) != len(matrix2):
        # Check matrix dimensions
        print 'Matrices must be m*n and n*p to multiply!'
    else:
        # Multiply if correct dimensions
        watch = Stopwatch()
        print 'mult1 start....'
        watch.Start()
        new_matrix = zero(len(matrix1),len(matrix2[0]))
        for i in range(len(matrix1)):
            for j in range(len(matrix2[0])):
                for k in range(len(matrix2)):
                    new_matrix[i][j] += matrix1[i][k]*matrix2[k][j]
        watch.Stop()
        print 'mult1 end.'
        print watch.ElapsedMilliseconds
        return new_matrix

from System import Array

def ListToArray(matrix):
    n = len(matrix)
    m = len(matrix[0])
    a = Array.CreateInstance(float, n, m)
    for i in range(n):
        for j in range(m):
            a[i,j] = matrix[i][j]
    return a


def mult2(matrix1, matrix2):

    N = len(matrix1)
    K = len(matrix2)
    M = len(matrix2[0])

    m1 = ListToArray(matrix1)
    m2 = ListToArray(matrix2)
    res = ListToArray(rand(len(matrix1), len(matrix2[0])))

    watch = Stopwatch()
    print 'mult2 start...'
    watch.Start()
    for i in range(N):
        for j in range(M):
            for k in range(K):
                res[i,j] += m1[i,k]*m2[k,j]
    watch.Stop()
    print 'mult2 ends.'
    print watch.ElapsedMilliseconds
    return res


if __name__ == '__main__':
    #a = rand(280,10304)
    #b = rand(10304,280)

    a = rand(280,10)
    b = rand(10,280)

    c = mult2(a, b)
    d = mult(a, b)

I want to try two big matrices (280 by 10304 and 10304 by 208), but both versions cannot produce a result within a short time.

Then I tried a much smaller one (as show in the code), I the result is following:

mult2 : 7902 ms
mult1 : 420 ms

indicating that using .NET array in IronPython is much slower than the python List.

Also notice that C# uses about ~12 seconds for the two big matrices. IronPython already spends a lot of them on the 10K times smaller case. I am not sure whether the IronPython setting in my computer is wrong or not, if not, IronPython is really slow for numerical code.

Answer 1

For your particular question, I think the problem is boxing - in IronPython, list items (and all other variables) are stored boxed, so only boxed values are operated on. CLR Array elements are not boxed, however, and thus IronPython will have to box them when they are extracted from the array, and then unbox them on the way back in. C# can operate on the unboxed values, and has a bunch of other optimizations to make arrays fast that IronPython doesn't have.

If you want fast numerical math, NumPy for IronPython might be a better bet.