Why does creating a list of tuples using list comprehension requires parentheses?

Question

As noted in Functional Programming HOWTO, to create a list of tuples using list comprehension, parentheses must be used. Particularly, this is expressed in the document in the following quote.

To avoid introducing an ambiguity into Python’s grammar, if expression is creating a tuple, it must be surrounded with parentheses.

So, as examples:

[x, y for x in seq1 for y in seq2]  # This is a syntex error
[(x, y) for x in seq1 for y in seq2]  # This is a correct expression of list of tuples using list comprehension

What ambiguity is avoided by forcing the use of parentheses in expressing a list of tuples using list comprehension?

Answer 1

After a lot of mailing list digging, I've found a pretty unambiguous statement that the parser was fine with it. The parentheses were made mandatory to make the meaning clearer. Here's a quote from Guido back in 2000 on the python-dev mailing list:

Don't worry. Greg Ewing had no problem expressing this in Python's own grammar, which is about as restricted as parsers come. (It's LL(1), which is equivalent to pure recursive descent with one lookahead token, i.e. no backtracking.)

Here's Greg's grammar:

atom: ... | '[' [testlist [list_iter]] ']' | ...
  list_iter: list_for | list_if
  list_for: 'for' exprlist 'in' testlist [list_iter]
  list_if: 'if' test [list_iter]

Note that before, the list syntax was '[' [testlist] ']'. Let me explain it in different terms:

The parser parses a series comma-separated expressions. Previously, it was expecting ']' as the sole possible token following this. After the change, 'for' is another possible following token. This is no problem at all for any parser that knows how to parse matching parentheses!

If you'd rather not support [x, y for ...] because it's ambiguous (to the human reader, not to the parser!), we can change the grammar to something like:

'[' test [',' testlist | list_iter] ']'

(Note that | binds less than concatenation, and [...] means an optional part.)

Also see the next response in the thread, where Greg Ewing runs

>>> seq = [1,2,3,4,5]
>>> [x, x*2 for x in seq]
[(1, 2), (2, 4), (3, 6), (4, 8), (5, 10)]

on an early version of the list comprehension patch, and it works just fine.