Why can't I get two lists from one list comprehension?

Question

So I have an array of the following form:

[(1, u'first_type', u'data_gid_1'), 
 (2, u'first_type', u'data_gid_2'), 
 (3, u'first_type', u'data_gid_3'), 
 (4, u'first_type', u'data_gid_4')]

Now I want to extract the first and the last element of each inside list into separate lists. So if I do:

>>> ids = [dat[0] for dat in all_data]
>>> gds = [dat[2] for dat in all_data]

This works as I expect it to. However I was trying to merge these two into one call, something like:

 (ids, gds) = [(dat[0], dat[2]) for dat in all_data]

This however fails with an: ValueError: too many values to unpack

So could anyone explain why this is happening and if what I am trying to do is even possible.

Regards, Bogdan

Answer 1

It doesn't work because the length of [(dat[0], dat[2]) for dat in all_data] is the same as the lenght of all_data, which is not the same length as the tuple (ids, gds).

Try this instead:

(ids, gds) = zip(*[(dat[0], dat[2]) for dat in all_data])

or even shorter:

(ids, gds) = zip(*all_data)[::2]

As noted in another answer, ids and gds will now be tuples, so if you need lists, do like this:

(ids, gds) = map(list, zip(*all_data)[::2])

---

The zip(*something) is a rather frequently occuring idiom in python. If you look at a list of lists as a matrix, i.e.

l = [[1, 2, 3],
     [4, 5, 6]]

Then zip(*l) transposes that matrix:

zip(*l) == [(1, 4),
            (2, 5),
            (3, 6)]

The * works like this: some_func(*some_list) unpacks some_list so that the function is in effect called with the elements of some_list as arguments. So zip(*l) is the same as zip([1, 2, 3], [4, 5, 6]). Here's the relevant part of the python tutorial.

zip acts like a zipper, hence the name, so it returns a list with these elements: a tuple of all the first elements of the given arguments, followed by a tuple of all the second elements, etc.

Answer 2

Code below will also do the trick:

data = [(1, u'first_type', u'data_gid_1'), 
 (2, u'first_type', u'data_gid_2'), 
 (3, u'first_type', u'data_gid_3'), 
 (4, u'first_type', u'data_gid_4')]

ids, gds = ([row[i] for row in data] for i in [0,2])

Answer 3

[(dat[0], dat[2]) for dat in all_data] means that you're creating a list of tuples (dat[0],dat[2]). It does not mean you have two lists, one of d[0] and the other of d[2].

You could simply use zip, but that will result in tuples. If you want lists, you'll have to apply list to the result:

(ids, gds) = map(list,zip(*[(dat[0], dat[2]) for dat in a]))