Python understanding list comprehension from Java background [duplicate]

Question

I come from a Java background and just started to work on Python. Most of the things are fairly easy to pick up but I am having hard time to understand one thing in the language which I just found out that is called list comprehension. What is this list comprehension in Python? How does this compare with language constructs found in Java? The problem is it's everywhere, nearly all the examples I found here and there use it.

For the following example, allow me to understand how this works.

[x**2 for x in range(10)]

And then there this.

[j + k for j in 'abc' for k in 'def']

Beyond that I also have seen things like this somewhere on Stackoverflow.

(x for x in (0,1,2,3,4))

Also things like this.

total = sum(x+y for x in (0,1,2,3) for y in (0,1,2,3) if x < y)

This started to get messy, could you please help me?

Answer 1

What is this list comprehension in Python?

First let’s start with the basic definition taken from the official Python documentation.

A list comprehension consists of brackets containing an expression followed by a for clause, then zero or more for or if clauses. The result will be a new list resulting from evaluating the expression in the context of the for and if clauses which follow it.

The problem is it's everywhere, nearly all the examples I found here and there use it.

List comprehension is a very flexible concept. It allows us to define lists as we know from the mathematics. Say we have a set S in which each element of S is a square of a number x and x could only takes values ranging from 0 to 10.

See the above definition. It took a paragraph to describe it. But there is a better way to describe it.

S = {x² : x in {0 ... 10}}

That’s why I love math, it is always to the point. Now remember your first example?

S = [x**2 for x in range(10)]

That’s the set we just defined. Neat right? That’s why it is used so much. (Don’t get confused with the x**2 syntax, because Python follows a weird syntax here which you probably might be familiar as x^2 from other languages.)

In Python you can iterate over pretty much anything. Which basically means you can easily iterate over each character of a string. Let’s look at the second example. It just iterates over the words ‘abc’ and ‘def’ and creates a list out of them.

lst = [j + k for j in 'abc' for k in 'def']

Notice that we assigned this to a list named lst. It was no coincidence. Python is the most humane programming language I have ever laid eyes on. So she will help you when you get stuck. Like this.

help(lst)

You can now see what you can do with lst. Ever got confused what lst is? You can check what it is via type.

print type(lst)

Before we move forward let’s talk a little bit about iterators and generators.

Iterators are objects that you can call next() method on. Like this.

iterator = iter([1,2,3,4])

Now we can print the first element of our iterator, like this.

print iterator.next()

Now we can talk about the generators. They are functions that generate iterators. There is however one other concept called generator expressions.

(x for x in (0,1,2,3,4))

This is a generator expression. A generator expressions is like a shortcut to build generators out of expressions similar to that of list comprehensions.

total = sum(x+y for x in (0,1,2,3) for y in (0,1,2,3) if x < y)

What above line does is to first create a generator using a generator expression and iterate over each element and sum those elements.