How can I speed up array generations in python?

Question

I'm thinking I need to use numpy or some other library to fill these arrays fast enough but I don't know much about it. Right now this operation takes about 1 second on a quad-core Intel PC, but I need it to be as fast as possible. Any help is greatly appreciated. Thanks!

import cv

class TestClass:

  def __init__(self):

    w = 960
    h = 540

    self.offx = cv.CreateMat(h, w, cv.CV_32FC1)
    self.offy = cv.CreateMat(h, w, cv.CV_32FC1)

    for y in range(h):
      for x in range(w):
        self.offx[y,x] = x
        self.offy[y,x] = y

Answer 1

My eight year old (slow) computer is able to create a list of lists the same size as your matrix in 127 milliseconds.

C:\Documents and Settings\gdk\Desktop>python -m timeit "[[x for x in range(960)]
 for y in range(540)]"
10 loops, best of 3: 127 msec per loop

I don't know what the cv module is and how it creates matrices. But maybe this is the cause of the slow code.

Numpy may be faster. Creating an array of (python int) 1s:

C:\Documents and Settings\gdk\Desktop>python -m timeit -s "from numpy import one
s" "ones((960, 540), int)"
100 loops, best of 3: 6.54 msec per loop

You can compare the timings for creating matrices using different modules to see if there is a benefit to changing: timeit module

Answer 2

You're generating a half million integers and creating over a million references while you're at it. I'd just be happy it only takes 1 second.

If you're doing this a lot, you should think about ways to cache the results.

Also, being on a quad-core anything doesn't help in a case like this, you're performing a serial operation that can only execute on one core at a time (and even if you threaded it, CPython can only be executing one pure-Python thread at a time due to the Global Interpreter Lock).