Convert partial function to method in python

Question

Consider the following (broken) code:

import functools                                                            
class Foo(object):                                                          
  def __init__(self):                                                 
    def f(a,self,b):                                            
      print a+b                                           
    self.g = functools.partial(f,1)                             
x=Foo()                                                                     
x.g(2)

What I want to do is take the function f and partially apply it, resulting in a function g(self,b). I would like to use this function as a method, however this does not currently work and instead I get the error

Traceback (most recent call last):
  File "test.py", line 8, in <module>
    x.g(2)
TypeError: f() takes exactly 3 arguments (2 given)

Doing x.g(x,2) however works, so it seem the issue is that g is considered a "normal" function instead of a method of the class. Is there a way to get x.g to behave like a method (i.e implicitly pass the self parameter) instead of a function?

Answer 1

There are two issues at hand here. First, for a function to be turned into a method it must be stored on the class, not the instance. A demonstration:

class Foo(object):
    def a(*args):
        print 'a', args

def b(*args):
    print 'b', args

Foo.b = b

x = Foo()

def c(*args):
    print 'c', args

x.c = c

So a is a function defined in the class definition, b is a function assigned to the class afterwards, and c is a function assigned to the instance. Take a look at what happens when we call them:

>>> x.a('a will have "self"')
a (<__main__.Foo object at 0x100425ed0>, 'a will have "self"')
>>> x.b('as will b')
b (<__main__.Foo object at 0x100425ed0>, 'as will b')
>>> x.c('c will only recieve this string')
c ('c will only recieve this string',)

As you can see there is little difference between a function defined along with the class, and one assigned to it later. I believe there is actually no difference as long as there is no metaclass involved, but that is for another time.

The second problem comes from how a function is actually turned into a method in the first place; the function type implements the descriptor protocol. (See the docs for details.) In a nutshell, the function type has a special __get__ method which is called when you perform an attribute lookup on the class itself. Instead of you getting the function object, the __get__ method of that function object is called, and that returns a bound method object (which is what supplies the self argument).

Why is this a problem? Because the functools.partial object is not a descriptor!

>>> import functools
>>> def f(*args):
...     print 'f', args
... 
>>> g = functools.partial(f, 1, 2, 3)
>>> g
<functools.partial object at 0x10042f2b8>
>>> g.__get__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'functools.partial' object has no attribute '__get__'

There are a number of options you have at this point. You can explicitly supply the self argument to the partial:

import functools
class Foo(object):                                                          
    def __init__(self):
        def f(self, a, b):                                            
            print a + b                                           
        self.g = functools.partial(f, self, 1)

x = Foo()
x.g(2)

...or you would imbed the self and value of a in a closure:

class Foo(object):                                                          
    def __init__(self):                                                 
        a = 1
        def f(b):                                            
            print a + b                                           
        self.g = f

x = Foo()
x.g(2)

These solutions are of course assuming that there is an as yet unspecified reason for assigning a method to the class in the constructor like this, as you can very easily just define a method directly on the class to do what you are doing here.

Edit: Here is an idea for a solution assuming the functions may be created for the class, instead of the instance:

class Foo(object):
    pass

def make_binding(name):
    def f(self, *args):
        print 'Do %s with %s given %r.' % (name, self, args)
    return f

for name in 'foo', 'bar', 'baz':
    setattr(Foo, name, make_binding(name))

f = Foo()
f.foo(1, 2, 3)
f.bar('some input')
f.baz()

Gives you:

Do foo with <__main__.Foo object at 0x10053e3d0> given (1, 2, 3).
Do bar with <__main__.Foo object at 0x10053e3d0> given ('some input',).
Do baz with <__main__.Foo object at 0x10053e3d0> given ().