Why does wrapping the random method of random.Random seem to affect the RNG?

Question

I tried to log calls to random() by putting a wrapper around it to print stuff. Surprisingly I noticed that I started getting different random values. I have created a small example to demonstrate the behavior:

Script main.py:

import random

def not_wrapped(seed):
    """ not wrapped """
    gen = random.Random()
    gen.seed(seed)

    # def wrappy(func):
    #     return lambda: func()
    # gen.random = wrappy(gen.random)

    gen.randint(1, 1)
    return gen.getstate()

def wrapped(seed):
    """ wrapped """
    gen = random.Random()
    gen.seed(seed)

    def wrappy(func):
        return lambda: func()
    gen.random = wrappy(gen.random)

    gen.randint(1, 1)
    return gen.getstate()

for s in range(20):
    print(s, not_wrapped(s) == wrapped(s))

Output of python3.7.5 main.py (same as python 3.6.9)

0 True
1 False
2 False
3 False
4 False
5 True
6 False
7 False
8 False
9 False
10 True
11 False
12 False
13 False
14 False
15 True
16 False
17 True
18 False
19 True

So as you see the state of the Random instance gen is different and it depends on if I wrap gen.random in wrappy or not.

If I call randint() twice the test will fail for all seeds 0-19. For Python 2.7.17 the wrapped and not_wrapped functions returns the same random value every time (prints True for every seed).

Does anyone know what is going on?

Python 3.6 online example: http://tpcg.io/inbKc8hK

On Python 3.8.2 on this online repl, the problem does not show: https://repl.it/@DavidMoberg/ExemplaryTeemingDos

(This question has been cross posted at: https://www.reddit.com/r/learnpython/comments/i597at/is_there_a_bug_in_randomrandom/)

Answer 1

This answer is about Python 3.6.

The state is only changed by the gen.randint(1, 1) call, so let's see what it calls under the hood.

1. Here is the implementation of random.Random.randint:

   def randint(self, a, b):
       """Return random integer in range [a, b], including both end points.
       """
   
       return self.randrange(a, b+1)
   

2. random.Random.randrange is right above, and it generates random numbers using random.Random._randbelow...

3. ...which is implemented right below randint, and checks whether the random method has been overridden!

Looks like _randbelow is the issue:

...
from types import MethodType as _MethodType, BuiltinMethodType as _BuiltinMethodType
...


def _randbelow(self, n, int=int, maxsize=1<<BPF, type=type,
               Method=_MethodType, BuiltinMethod=_BuiltinMethodType):
    "Return a random int in the range [0,n).  Raises ValueError if n==0."

    random = self.random
    getrandbits = self.getrandbits

    # CHECKS IF random HAS BEEN OVERRIDDEN!

    # If not, this is the default behaviour:

    # Only call self.getrandbits if the original random() builtin method
    # has not been overridden or if a new getrandbits() was supplied.
    if type(random) is BuiltinMethod or type(getrandbits) is Method:
        k = n.bit_length()  # don't use (n-1) here because n can be 1
        r = getrandbits(k)          # 0 <= r < 2**k
        while r >= n:
            r = getrandbits(k)
        return r

    # OTHERWISE, it does something different!

    # There's an overridden random() method but no new getrandbits() method,
    # so we can only use random() from here.

    # And then it goes on to use `random` only, no `getrandbits`!

(For reference, getrandbits is implemented in C here, and random is also implemented in C here.)

So there's the difference: the randint method ends up behaving differently depending on whether random has been overridden or not.