Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Differences between functools.partial and a similar lambda?

In Python, suppose I have a function f that I want to pass around with some secondary arguments (assume for simplicity that it's just the first argument that remains variable).

What are the differences between doing it these two ways (if any)?

# Assume secondary_args and secondary_kwargs have been defined

import functools

g1 = functools.partial(f, *secondary_args, **secondary_kwargs)
g2 = lambda x: f(x, *secondary_args, **secondary_kwargs)

In the doc page for partial, for example, there is this quote:

partial objects defined in classes behave like static methods and do not transform into bound methods during instance attribute look-up.

Will the lambda-method suffer from this if used to make a class method from arguments supplied to the class (either in the constructor or through a function later on)?

like image 543
ely Avatar asked Aug 06 '12 12:08

ely


People also ask

What is Functools partial?

You can create partial functions in python by using the partial function from the functools library. Partial functions allow one to derive a function with x parameters to a function with fewer parameters and fixed values set for the more limited function.

What is the difference lambda function?

Lambda Function, also referred to as 'Anonymous function' is same as a regular python function but can be defined without a name. While normal functions are defined using the def keyword, anonymous functions are defined using the lambda keyword. However,they are restricted to single line of expression.

What is Functools?

Functools module is for higher-order functions that work on other functions. It provides functions for working with other functions and callable objects to use or extend them without completely rewriting them.

Are lambdas and anonymous functions the same?

Anonymous functions, lambda expressions, or function literals are all the same thing. Lambda (or \lambda) is the name given to anonymous functions in some languages like Python. These are functions not bound by an explicit identifier.


2 Answers

  1. A lambda function has the same type as a standard function, so it will behave like an instance method.

  2. The partial object in your example can be called like this:

    g1(x, y, z)
    

    leading to this call (not valid Python syntax, but you get the idea):

    f(*secondary_args, x, y, z, **secondary_kwargs)
    

    The lambda only accepts a single argument and uses a different argument order. (Of course both of these differences can be overcome – I'm just answering what the differences between the two versions you gave are.)

  3. Execution of the partial object is slightly faster than execution of the equivalent lambda.

like image 183
Sven Marnach Avatar answered Oct 06 '22 08:10

Sven Marnach


Summary

The practical differences between lambda and functools.partial in the common use cases seems to be

  • functools.partial needs an import, lambda does not.
  • The function definition for functions created with functools.partial is visible just by printing the created function. The functions created with lambda should be inspected with inspect.getsource().

These were found to be practically identical for lambda and functools.partial

  • Speed
  • Tracebacks

Speed (lambda vs functools.partial)

I think that tests and real data speaks louder than just guesses about which one is faster than the another.

Looks like that there is no statistical proof for speed difference between lambda and functools.partial. I ran different tests with different amount of repetitions, getting slightly different results each time; any of the three approaches could be the fastest. The speeds were identical with 95% (2 sigma) confidence. Here are some numerical results*

# When functions are defined beforehand
In [1]: timeit -n 1000 -r 1000 f_partial(data)
23.6 µs ± 2.92 µs per loop (mean ± std. dev. of 1000 runs, 1000 loops each)

In [2]: timeit -n 1000 -r 1000 f_lambda(data)
22.6 µs ± 2.6 µs per loop (mean ± std. dev. of 1000 runs, 1000 loops each)

# When function is defined each time again
In [3]: timeit -n 1000 -r 1000 (lambda x: trim_mean(x, 0.1))(data)
22.6 µs ± 1.98 µs per loop (mean ± std. dev. of 1000 runs, 1000 loops each)

In [4]: timeit -n 1000 -r 1000 f_lambda = lambda x: trim_mean(x, 0.1); f_lambda(data)
23.7 µs ± 3.89 µs per loop (mean ± std. dev. of 1000 runs, 1000 loops each)

In [5]: timeit -n 1000 -r 1000 f_partial = partial(trim_mean, proportiontocut=0.1); f_partial(data)
24 µs ± 3.38 µs per loop (mean ± std. dev. of 1000 runs, 1000 loops each)

Tracebacks

I also tried running the f_lambda and f_partial using list with string element inserted, and the tracebacks were equal (except for the very first entry, of course). So there is no difference there.

Inspecting the source code

  • The function definition for functions created with functools.partial is visible just by printing the created function. The functions created with lambda should be inspected with inspect.getsource().
# Can be inspected with just printing the function
In [1]: f_partial
Out[1]: functools.partial(<function trim_mean at 0x000001463262D0D0>, proportiontocut=0.1)

In [2]: print(f_partial)
functools.partial(<function trim_mean at 0x000001463262D0D0>, proportiontocut=0.1)

# Lambda functions do not show the source directly
In [3]: f_lambda
Out[3]: <function __main__.<lambda>(x)>

# But you can use inspect.getsource()
In [4]: inspect.getsource(f_lambda)
Out[4]: 'f_lambda = lambda x: trim_mean(x, 0.1)\n'

# This throws a ValueError, though.
In [5]: inspect.getsource(f_partial)

Appendix

* Setup used in the tests

from functools import partial
from scipy.stats import trim_mean
import numpy as np
data = np.hstack((np.random.random(1000), np.random.random(50)*25000))

f_lambda = lambda x: trim_mean(x, 0.1)
f_partial = partial(trim_mean, proportiontocut=0.1)

The tests were performed on Python 3.7.3 64-bit (Windows 10).

like image 31
np8 Avatar answered Oct 06 '22 09:10

np8