What does this function do? (Python iterators)

Question

def partition(n, iterable):
   p = izip_longest(*([iter(iterable)] * n))
   r = []
   for x in p:
       print(x) #I added this
       s = set(x)
       s.discard(None)
       r.append(list(s))
   return r

This is actually in a job posting on SO and being a newbie I thought it was interesting. So you get output like the following:

partition(5, L)
(1, 2, 3, 4, None)
Out[86]: [[1, 2, 3, 4]]

To me this is already confusing because I thought izip_longest(*([iter(iterable)] * n))would run the izip_longest function on a list of n identical iterators so I would have expected first an output of (1,1,1,1,1) and then an output of (2,2,2,2,2) and so on.

So short version of my question, is what's going on with this line:

 p = izip_longest(*([iter(iterable)] * n))

Parsing it I would have thought [iter(iterable)]*n creates a list of length n of identical iterables all pointing to the same thing - that's what it does on the command line, but that doesn't seem to be what it does here based on the output printed above.

Also I thought the * at the beginning ...longest(*... was there since the list is of unknown length but I don't think that entirely makes sense. What is that first * symbol doing inside the function call? Doesn't seem like it's simply indicating an unknown length list of arguments...

So at the end of the day I'm completely lost. Can someone walk me through this syntax?

Thanks a lot for any input!

---

Thanks for all the helpful answers, everyone. I am not sure if I am tacking on an answer or a question here, but it seems to me this list comprehension will do the same thing for lists and tuples (I realize iterators would also apply to dictionaries, custom classes, other things...)

[L[i*n:(i+1)*n] for i in range(int(ceil(len(L)/float(n)))) ]

Answer 1

Given:

>>> li
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20]

There is a common Python idiom of using zip in combination with iter and * operator to partition a list a flat list into a list of lists of n length:

>>> n=3
>>> zip(*([iter(li)] * n))
[(0, 1, 2), (3, 4, 5), (6, 7, 8), (9, 10, 11), (12, 13, 14), (15, 16, 17), (18, 19, 20)]

However, if n is not an even multiple of the overall length, the final list is truncated:

>>> n=4
>>> zip(*([iter(li)] * n))
[(0, 1, 2, 3), (4, 5, 6, 7), (8, 9, 10, 11), (12, 13, 14, 15), (16, 17, 18, 19)]

You can use izip_longest to use the complete list filled in with a selected value for the incomplete sub lists:

>>> list(izip_longest(*([iter(li)] * n)))
[(0, 1, 2, 3), (4, 5, 6, 7), (8, 9, 10, 11), (12, 13, 14, 15), (16, 17, 18, 19), (20, None, None, None)]

Answer 2

iter(my_list) converts a list into an iterable (that is one where elements are consumed as they are seen)

[my_iter]*5 creates a new list of [my_iter,my_iter,my_iter,my_iter,my_iter] in which all the my_iter's point to the same exact iterator

zip(*[my_iter,my_iter,my_iter,my_iter,my_iter]) 

is the same as

zip(my_iter,my_iter,my_iter,my_iter,my_iter)

(the splat simply unpacks a list/tuple) which basically just returns a 5xlen(my_list)//5 2d list

you could simplify it with normal zip

#this method will have no `None` entries
new_list_partitioned = zip(*[iter(big_list)]*N) + [big_list[-(len(big_list)%N):],]