Can we method chain on lists?

Question

I come from Ruby and you can method chain very easily. Let's look at an example. If I want to select all even nums from a list and add 5 to it. I would do something like this in Ruby.

nums = [...]
nums.select {|x| x % 2 == 0 }.map { |x| x + 5 }

In Python that becomes

nums = [...]
list(map(lambda x: x + 5, filter(lambda x: x % 2 == 0, nums)))

The Python syntax looks horrible. I tried to Google and didn't really find any good answers. All I saw was how you can achieve something like this with custom objects but nothing to process lists this way. Am I missing something?

When in a debugging console, it used to be extremely helpful to get som ActiveRecord objects in an array and I could just chain methods to process the entities to debug things. With Python, it almost seems like too much work.

Answer 1

In Ruby, every enumerable object includes the Enumerable interface, which is why we get all of those helpful methods like you mention. But in Python, there's no common superclass for iterables. An iterable is literally defined as "a thing which supports __iter__", and while there is an abstract class called Iterable which pretends to be a superclass of all iterables, it doesn't actually provide any methods and it doesn't sit in the inheritance chain of all iterables (it overrides the behavior of isinstance and issubclass using the magic of dunder methods, the same way you can override + by writing __add__).

The Alakazam library implements exactly this feature. (Disclosure: I am the creator and maintainer of this library, but it does exactly what you're asking for, so I'll mention it here)

Alakazam provides the Alakazam class, which wraps any Python iterable and provides, as methods, all of the built-in Python sequence methods, all of the itertools module, and some other useful stream-oriented methods that aren't included in Python by default. Consider your example from above

nums.select {|x| x % 2 == 0 }.map { |x| x + 5 }

In Python, that looks like

list(map(lambda x: x + 5, filter(lambda x: x % 2 == 0, nums)))

With Alakazam, that looks like

zz.of(nums).filter(lambda x: x % 2 == 0).map(lambda x: x + 5).list()

or, using Alakazam's lambda syntax

zz.of(nums).filter(_1 % 2 == 0).map(_1 + 5).list()

Whenever reasonable, Alakazam's methods like filter and map are lazy to match Python's behavior, so we still need to write list() at the end to consume the iterable and produce a single list result.

Answer 2

As noted in comments, this Ruby code:

nums = [...]
nums.select {|x| x % 2 == 0 }.map { |x| x + 5 }

Note: why not use #even??

nums = [...]
nums.select {|x| x.even? }.map { |x| x + 5 }

Or even:

nums = [...]
nums.select(&:even?).map { |x| x + 5 }

But nitpicks aside, this can be expressed in Python using a list comprehension, which is very clean.

nums = [...]
[x + 5 for x in nums if x % 2 == 0]

Now a list comprehension eagerly generates a full list. Imagine an original list like [1, 2, 3, 4, 5, 6, 7, 8]. The list comprehension would give us [2, 4, 6, 8]. The data set is trivial.

But imagine that nums is list(range(100_000_000)). Not a trivial data set. Applying this list comprehension to the whole thing will take a lot of time, even if we only need the first four values.

But a generator expression lets us lazily generate the values we need.

from itertools import islice

nums = range(100_000_000)
evens_plus_five = (x + 5 for x in nums if x % 2 == 0)

list(islice(evens_plus_five, 0, 5, 1))

As suggested in comments, this lazy evaluation advantage on large data sets can be gained in Ruby quite readily using #lazy and ranges.

nums = (1..100_000_000)
nums.lazy.select(&:even?).map { |x| x + 5 }.take(5).to_a

And if you're using Ruby 3, let's make that block even cleaner.

nums = (1..100_000_000)
nums.lazy.select(&:even?).map { _1 + 5 }.take(5).to_a