Is it possible to split a sequence of pandas commands across multiple lines?

Question

I have a long string of pandas chained commands, for example:

df.groupby[['x','y']].apply(lambda x: (np.max(x['z'])-np.min(x['z']))).sort_values(ascending=False)

And I would like to be able to present it across multiple lines but still as a one liner (without saving results to a temporary object, or defining the lambda as a function)

an example of how I would like it to look:

df.groupby[['x','y']]
.apply(lambda x: (np.max(x['z'])-np.min(x['z'])))
.sort_values(ascending=False)

Is it possible to do so? (I know '_' has this functionality in python, but it doesn't seem to work with chained commands)

Answer 1

In python you can continue to the next line by ending your line with a reverse slash or by enclosing the expression in parenthesis.

df.groupby[['x','y']] \
.apply(lambda x: (np.max(x['z'])-np.min(x['z']))) \
.sort_values(ascending=False)

or

(df.groupby[['x','y']]
.apply(lambda x: (np.max(x['z'])-np.min(x['z'])))
.sort_values(ascending=False))

Answer 2

The preferred way of wrapping long lines is by using Python's implied line continuation inside parentheses, brackets and braces. Long lines can be broken over multiple lines by wrapping expressions in parentheses. These should be used in preference to using a backslash for line continuation

from https://www.python.org/dev/peps/pep-0008/#id19

So may be better:

df.groupby[['x', 'y']].apply(
    lambda x: (np.max(x['z'])-np.min(x['z']))
).sort_values(ascending=False)

The last printed expression variable "_" is known only in the Python console, so without explicit attribution cannot be used for that purpose in a script/module.

Answer 3

Since this has the nature of a command, I would probably format it close to your example, like this:

df.groupby[['x','y']] \
    .apply(lambda x: np.max(x['z'])-np.min(x['z'])) \
    .sort_values(ascending=False)

It took me a long time to realize I could break these expressions before the dots, which is often more readable than breaking inside the parentheses (same goes for "some long string".format()).

If this were more like an expression evaluation, I'd wrap the whole thing in parentheses, which is considered more "Pythonic" than line continuation markers:

var = (
    df.groupby[['x','y']]
        .apply(
            lambda x: np.max(x['z'])-np.min(x['z'])
        ) 
        .sort_values(ascending=False)
)

Update Since writing this, I've moved away from backslashes for line continuation whenever possible, including here, where it's not meaningful to chain the operations without assigning it to a variable or passing it to a function. I've also switched to using one level of indentation for each level of nesting inside parentheses or brackets, to avoid going to deep and/or getting a wiggly effect. So I would now write your expression like this:

 var = (
    df
    .groupby[['x','y']]
    .apply(
        lambda x: np.max(x['z']) - np.min(x['z'])
    ) 
    .sort_values(ascending=False)
)