Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How do I efficiently filter computed values within a Python list comprehension?

Tags:

The Python list comprehension syntax makes it easy to filter values within a comprehension. For example:

result = [x**2 for x in mylist if type(x) is int] 

Will return a list of the squares of integers in mylist. However, what if the test involves some (costly) computation and you want to filter on the result? One option is:

result = [expensive(x) for x in mylist if expensive(x)] 

This will result in a list of non-"false" expensive(x) values, however expensive() is called twice for each x. Is there a comprehension syntax that allows you to do this test while only calling expensive once per x?

like image 249
Nick Avatar asked Sep 24 '08 22:09

Nick


People also ask

Which is faster filter or list comprehension?

1 Answer. Actually, list comprehension is much clearer and faster than filter+lambda, but you can use whichever you find easier.

How do you filter a list in Python?

Python has a built-in function called filter() that allows you to filter a list (or a tuple) in a more beautiful way. The filter() function iterates over the elements of the list and applies the fn() function to each element. It returns an iterator for the elements where the fn() returns True .

Are Python filters efficient?

For large lists with one million elements, filtering lists with list comprehension is 40% faster than the built-in filter() method. The reason is the efficient implementation of the list comprehension statement.

What is faster than list comprehension in Python?

For loops are faster than list comprehensions to run functions.


2 Answers

Came up with my own answer after a minute of thought. It can be done with nested comprehensions:

result = [y for y in (expensive(x) for x in mylist) if y] 

I guess that works, though I find nested comprehensions are only marginally readable

like image 187
Nick Avatar answered Oct 01 '22 22:10

Nick


If the calculations are already nicely bundled into functions, how about using filter and map?

result = filter (None, map (expensive, mylist)) 

You can use itertools.imap if the list is very large.

like image 40
John Millikin Avatar answered Oct 01 '22 23:10

John Millikin