Numba error when attempting to instantiate NumPy RandomState

Question

I have a class that I'd like to speed up with Numba. The class employs a "random number generator" for each instance by simply creating an instance of NumPy's RandomState with a particular seed (thus I can replicate my work later). When I use Numba's autojit, I get a strange error which doesn't arise in "regular" Python.

Fortunately this behavior is extremely simple to replicate. Here's a simple example which illustrates the error:

from numpy.random import RandomState
from numba import autojit

# ------- This works in "regular Python" ------------

class SillyClass1(object):
    def __init__(self, seed):
        self.RNG = RandomState(seed)
    def draw_uniform(self):
        return self.RNG.uniform(0,1)

test1 = SillyClass1(123456)

test1.draw_uniform()

# Output:
# 0.12696983303810094


# The following code -- exactly the same as above, but with the @autojit 
# decorator, doesn't work, and throws an error which I am having a hard  
# time understanding how to fix:

@autojit
class SillyClass2(object):
    def __init__(self, seed):
        self.RNG = RandomState(seed)
    def draw_uniform(self):
        return self.RNG.uniform(0,1)

test2 = SillyClass2(123456)

test2.draw_uniform()

# Output:
#
# ValueError                                Traceback (most recent call last)
# <ipython-input-86-a18f95c11a1b> in <module>()
#      10 
#      11 
# ---> 12 test2 = SillyClass2(123456)
#      13 
#      14 test2.draw_uniform()
# 
# ...
# 
# ValueError: object of too small depth for desired array

I'm using the Anaconda distributionon Ubuntu 13.10.

Any thoughts?

Edit: I found a work-around, which is to simply use Python's standard "random.Random" instead of NumPys' "numpy.random.RandomState"

Example:

from random import Random 
@autojit
class SillyClass3(object):
    def __init__(self, seed):
        self.RNG = Random(seed)
    def draw_uniform(self):
        return self.RNG.uniform(0,1)

test3 = SillyClass3(123456)

test3.draw_uniform()

# Output:
# 0.8056271362589

This works for my immediate application (although other problems arose immediately, hurray).

However, this fix will not work for future algorithms for which I know I will need to use numpy.random.RandomState. So my question still stands -- does anyone have any insight regarding the original error, and/or workarounds for using numy.random.RandomState in Numba?

Answer 1

A late answer, but the issue is that Numba does not support Numpy's RandomState object (as of July, 2019). Neither does it support setting random states using the Python standard library.

However, given that np.random.seed is supported by Numba, you can use a numpy's PRNG within a jitted function by passing a seed parameter into the function and then calling the np.random.seed(seed) method therein. From the Numba documentation:

Numba supports top-level functions from the numpy.random module, but does not allow you to create individual RandomState instances. The same algorithms are used as for the standard random module (and therefore the same notes apply), but with an independent internal state: seeding or drawing numbers from one generator won’t affect the other.

Code Example

import random, numba, numpy

@numba.njit
def random_func(seed: int = 1234) -> str:
    seeded_prng = numpy.random.seed(seed)
    # Do random stuff
    random_computation = numpy.random.uniform(0, 1)
    return random_computation

# Example Invocation
seed = 1337
random_result = random_func(seed)
print(random_result)

Which should yield:

0.2620246750155817

If you'd like to have a different random state for each class instance, you'll need to have a separate thread for each class because the PRNGs are tied to the thread they are instantiated in. Fortunately, Numba handles this automatically provided each class occurs in its own thread:

Since version 0.28.0, the generator is thread-safe and fork-safe. Each thread and each process will produce independent streams of random numbers.

Finally, if you are targeting CUDA, there are a number of methods that allow you to work directly with a GPU enabled PRNG that could be handy. See this page from the Numba documentation.