How can random.shuffle result in a KeyError?

Question

I just got the error:

Traceback (most recent call last):
  File "./download_documents.py", line 153, in <module>
    paragraphs, used_pages = find_pages(lang, to_extract)
  File "./download_documents.py", line 67, in find_pages
    random.shuffle(page_titles_queue)
  File "/usr/lib/python2.7/random.py", line 291, in shuffle
    x[i], x[j] = x[j], x[i]
KeyError: 1

Which confuses me quite a bit.

1. random.shuffle seems to work on zero-element lists and on one-element lists.
2. page_titles_queue is a list of tuples.
3. Two lines after the random.shuffle(page_titles_queue), there is page_titles_queue.pop(), but that should not affect the shuffle. Right?

So what are possible reasons for the KeyError?

I use Python 2.7.12 on Ubuntu 16.04.

Answer 1

random.shuffle just exchanges items, the line where the exception happened makes this perfectly clear:

x[i], x[j] = x[j], x[i]

Where x is the "sequence" that was passed in. In this case i and j will be values in the range range(0, len(x)) and if any of these i or j isn't present in the "sequence" it will throw an Exception. In your case it's very likely given that it throws a KeyError:

>>> import random
>>> d = {i: i for i in range(7, 10)}
>>> random.shuffle(d)
KeyError: 3

However it works by exchanging the values in case the dictionary contains exactly the keys that make up the range(0, len(x)):

>>> d = {i: i for i in range(10)}
>>> random.shuffle(d)
>>> d
{0: 7, 1: 9, 2: 3, 3: 4, 4: 0, 5: 2, 6: 1, 7: 6, 8: 8, 9: 5}

If one or multiple keys are missing it could work or it could throw an Exception. That depends on which random numbers will be drawn:

d = {i: i for i in range(1, 10)}
random.shuffle(d)   # works sometimes, but sometimes it throws the KeyError