initialize a float32 matrix with random numbers in numpy/scipy

Question

scipy.random.rand() and other functions in the same package all produce arrays of float64 as output (at least for python 2.7.3 64-bit on Mac OS, scipy version 0.12.0).

What I want is a rather large (N gigabytes) randomly initialized matrix of float32. Is there an easy way to produce one directly, rather than allocating double space for float64 then converting down to 32 bits?

Answer 1

I would preallocate the array, then copy in batches of random float64s as Warren Weckesser recommends in the comments.

If you're in for a hack, here's ten floats generated using uniform random bits:

>>> bytes_per_float = np.float32(0).nbytes  # ugly, I know
>>> np.frombuffer(random.bytes(10 * bytes_per_float), dtype=np.float32)
array([ -3.42894422e-23,  -3.33389699e-01,  -7.63695071e-26,
         7.02152836e-10,   3.45816648e-18,   2.80226597e-09,
        -9.34621269e-10,  -9.75820352e+08,   2.95705402e+20,
         2.57654391e+25], dtype=float32)

Of course, these don't follow any nice distribution, the array might contain NaN or Inf, and the code might actually crash on some non-x86 machines due to aligment problems.