Generate normal random numbers in float32/16 in-place

Question

In Numpy/Scipy, how can I generate random numbers from a normal distribution with a specified (float) dtype? In my case I need float32 and float16.

Since the array is quite large, I don't want to convert the array after the sampling.

For example:

a = np.random.normal(1e7).astype('float16')

does the job but since it need a temporary float64 array it uses 4x the RAM than a direct float16 sampling.

Answer 1

As @JChoi points out, the standard_normal method of the Generator class in numpy.random accepts a dtype argument that allows the direct generation of numpy.float32 samples. It only accepts numpy.float32 and numpy.float64 for dtype, so it won't help with numpy.float16.

I don't know of a random number generator in numpy or scipy that generates 16 bit floats natively.

To avoid the large temporary, you could generate the values in batches. For example, the following creates an array of 10000000 samples of float16 values.

In [125]: n = 10000000  # Number of samples to generate

In [126]: k = 10000     # Batch size

In [127]: a = np.empty(n, dtype=np.float16)

In [128]: for i in range(0, n, k):
   .....:     a[i:i+k] = np.random.normal(loc=0, scale=1, size=k)
   .....:

Answer 2

standard_normal in Random Generator now supports dtype argument.

>>> import numpy as np
>>> rng = np.random.default_rng()
>>> rng.standard_normal(10, dtype=np.float32)
array([ 1.0513772 ,  0.471835  ,  0.22101597, -0.5053934 , -0.33512875,
       -0.20794971, -0.2699952 , -1.2080398 , -0.16769128,  1.3502929 ],
      dtype=float32)