Fastest way to create an array in Python [duplicate]

Question

I want to create a 3D array in Python, filled with -1.

I tested these methods:

import numpy as np
l = 200
b = 100
h = 30

%timeit grid = [[[-1 for x in range(l)] for y in range(b)] for z in range(h)]
1 loops, best of 3: 458 ms per loop

%timeit grid = -1 * np.ones((l, b, h), dtype=np.int)
10 loops, best of 3: 35.5 ms per loop

%timeit grid = np.zeros((l, b, h), dtype=np.int) - 1
10 loops, best of 3: 31.7 ms per loop

%timeit grid = -1.0 * np.ones((l, b, h), dtype=np.float32)
10 loops, best of 3: 42.1 ms per loop

%%timeit
grid = np.empty((l,b,h))
grid.fill(-1.0)
100 loops, best of 3: 13.7 ms per loop

So obviously, the last one is the fastest. Does anybody has an even faster method or at least less memory intensive? Because it runs on a RaspberryPi.

Answer 1

The only thing I can think to add is that any of these methods will be faster with the dtype argument chosen to take up as little memory as possible.

Assuming you need no more space that int8, the method suggested by @RutgerKassies in the comments took this long on my system:

%timeit grid = np.full((l, b, h), -1, dtype=np.int8)
1000 loops, best of 3: 286 µs per loop

For comparison, not specifying dtype (defaulting to int32) took about 10 times longer with the same method:

%timeit grid = np.full((l, b, h), -1)
100 loops, best of 3: 3.61 ms per loop

Your fastest method was about as fast as np.full (sometimes beating it):

%%timeit
grid = np.empty((l,b,h))
grid.fill(-1)
100 loops, best of 3: 3.51 ms per loop

or, with dtype specified as int8,

1000 loops, best of 3: 255 µs per loop

Edit: This is probably cheating, but, well...

%timeit grid = np.lib.stride_tricks.as_strided(np.array(-1, dtype=np.int8), (l, b, h), (0, 0, 0))
100000 loops, best of 3: 12.4 us per loop

All that's happening here is that we begin with an array of length one, np.array([-1]), and then fiddle with the stride lengths so that grid looks exactly like an array with the required dimensions.

If you need an actual array, you can use grid = grid.copy(); this makes the creation of the grid array about as fast as the quickest approaches suggested on elsewhere this page.