Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to randomly shuffle a list that has more permutations than the PRNG's period?

I have a list with around 3900 elements that I need to randomly permute to produce a statistical distribution. I looked around and found this Maximal Length of List to Shuffle with Python random.shuffle that explains that the period of the PRNG in Python is 2**19937-1, which leads to a list with a maximum length of 2080 before it becomes impossible to generate all possible permutations. I am only producing 300-1000 permutations of the list so it unlikely that I will be producing duplicate permutations, however, since this is producing a statistical distribution I would like to have all possible permutations as potential samples.

like image 224
Darwin Avatar asked Dec 07 '15 17:12

Darwin


People also ask

How do you randomize the order of items in a list?

Python Random shuffle() Method The shuffle() method takes a sequence, like a list, and reorganize the order of the items. Note: This method changes the original list, it does not return a new list.

How do you randomly permute a list in Python?

The random. sample() function returns the random list with the sample size you passed to it. For example, sample(myList, 3) will return a list of 3 random items from a list. If we pass the sample size the same as the original list's size, it will return us the new shuffled list.


1 Answers

There are longer-period PRNGs than MT, but they are hard to find.

To get all 3090! combinations, you need 40,905 bits of entropy. That's about 5kb. You should be able to grab a chunk of bytes that size from someplace like random.org many times with no problem. To get precisely balanced, you'll have to add some and do rejection sampling. I.e., grab 12 bits at a time (0..4095), and reject numbers higher than your current loop index. That might inflate the number of bits needed, but probably not beyond 8kb.

like image 197
Lee Daniel Crocker Avatar answered Oct 30 '22 01:10

Lee Daniel Crocker