Why can you loop through an implicit tuple in a for loop, but not a comprehension in Python?

Question

Is there a reason why looping through an implicit tuple in a for loop is okay, but when you do it in a comprehension you get a syntax error?

For example:

for i in 'a','b','c': 
    print(i)

'a'
'b'
'c'

But in a comprehension:

>>> [i for i in 'a','b','c']
  File "<stdin>", line 1
    [i for i in 'a','b','c']
                   ^
SyntaxError: invalid syntax

Is there a reason for this? I wasn't sure about the correct terminology, so my searches yielded nothing useful.

Update:

Per the comments, this syntax is valid for Python 2.x, but not for Python 3.x.

Answer 1

Because the for i in in the first code is a different syntactical construction than the for i in in the second code.

The first case is a for statement, which has the grammar:

for_stmt ::=  "for" target_list "in" expression_list ":" suite
             ["else" ":" suite]

'a', 'b', 'c' is most definitely an expression_list, so that works out.

In the second case, however, the inline for inside square brackets forces the code to be interpreted as a list comprehension, and in Python 3, list comprehensions must have the syntax:

comprehension ::=  expression comp_for
comp_for      ::=  "for" target_list "in" or_test [comp_iter]
comp_iter     ::=  comp_for | comp_if
comp_if       ::=  "if" expression_nocond [comp_iter]

Note that the part after the in must be an or_test, yet comma-delimited expressions create expression lists, and an expression list cannot be an or_test --- or, put another way, or has higher precedence than comma. Python thus thinks the comprehension ends at the comma, so that the three elements of the list are:

i for i in 'a'
'b'
'c'

which (unless you put the i for i in 'a' in parentheses) is obviously invalid.

As to why this works in Python 2 ... I'm still looking.

Answer 2

This changed in Python3, mainly in order to make list comprehensions more consistent with generator expressions.

With for-loops and list comprehensions, there is no ambiguity when using a tuple with no parentheses, because the former is always terminated by a colon, and the latter by either a closing bracket or a for/if keyword.

However, part of the design of generator expressions requires that they can be used "bare" as function arguments:

>>> list(i for i in range(3))
[0, 1, 2]

which creates some ambiguity for unparenthesized tuples, because any commas may introduce a new argument:

>>> list(i for i in 0, 1, 2)
  File "<stdin>", line 1
SyntaxError: Generator expression must be parenthesized if not sole argument

So tuples must always be parenthesized in generator expressions, and the same restriction now also applies to list comprehensions in order to preserve consistency.

PS:

Guido van Rossum wrote a article that spells out all the details on this subject in his History of Python blog:

• From List Comprehensions to Generator Expressions