How to apply class decorator at base of all decorators on methods

Question

I am using this way of decorating all methods

import inspect

def decallmethods(decorator, prefix='test_'):
  def dectheclass(cls):
    for name, m in inspect.getmembers(cls, inspect.ismethod):
      if name.startswith(prefix):
        setattr(cls, name, decorator(m))
    return cls
  return dectheclass


@decallmethods(login_testuser)
class TestCase(object):
    def setUp(self):
        pass

    def test_1(self):
        print "test_1()"

    def test_2(self):
        print "test_2()"

This is working but it applies at the top , if i have other decorators.

I mean

Now the result is

@login_testuser
@other
def test_2(self):
    print "test_2()"

But i want

@other
@login_testuser
def test_2(self):
    print "test_2()"

Answer 1

This is most certainly a bad idea, but what you want to do can be done in some extent, and this is going to take a lot of time to explain. First off, rather than thinking of decorators as a syntax sugar, think of them as what they really are: a function (that is a closure) with a function that exist inside it. Now this is out of the way, supposed we have a function:

def operation(a, b):
    print('doing operation')
    return a + b

Simply it will do this

>>> hi = operation('hello', 'world')
doing operation
>>> print(hi)
helloworld

Now define a decorator that prints something before and after calling its inner function (equivalent to the other decorator that you want to decorator later):

def other(f):  
    def other_inner(*a, **kw):
        print('other start')
        result = f(*a, **kw)
        print('other finish')
        return result
    return other_inner

With that, build a new function and decorator

@other
def o_operation(a, b):
    print('doing operation')
    return a + b

Remembering, this is basically equivalent to o_operation = other(operation)

Run this to ensure it works:

>>> r2 = o_operation('some', 'inner')
other start
doing operation
other finish
>>> print(r2)
someinner

Finally, the final decorator you want to call immediately before operation but not d_operation, but with your existing code it results in this:

def inject(f):
    def injected(*a, **kw):
        print('inject start')
        result = f(*a, **kw)
        print('inject finish')
        return result
    return injected

@inject
@other
def i_o_operation(a, b):
    print('doing operation')
    return a + b

Run the above:

>>> i_o_operation('hello', 'foo')
inject start
other start
doing operation
other finish
inject finish
'hellofoo'

As mentioned decorators are really closures and hence that's why it's possible to have items inside that are effectively instanced inside. You can reach them by going through the __closure__ attribute:

>>> i_o_operation.__closure__
(<cell at 0x7fc0eabd1fd8: function object at 0x7fc0eabce7d0>,)
>>> i_o_operation.__closure__[0].cell_contents
<function other_inner at 0x7fc0eabce7d0>
>>> print(i_o_operation.__closure__[0].cell_contents('a', 'b'))
other start
doing operation
other finish
ab

See how this effectively calls the function inside the injected closure directly, as if that got unwrapped. What if that closure can be replaced with the one that did the injection? For all of our protection, __closure__ and cell.cell_contents are read-only. What needs to be done is to construct completely new functions with the intended closures by making use of the FunctionType function constructor (found in the types module)

Back to the problem. Since what we have now is:

i_o_operation = inject(other(operation))

And what we want is

o_i_operation = other(inject(operation))

We effectively have to somehow strip the call to other from i_o_operation and somehow wrap it around with inject to produce o_i_operation. (Dragons follows after the break)

---

First, construct a function that effectively calls inject(operation) by taking the closure to level deep (so that f will contain just the original operation call) but mix it with the code produced by inject(f):

i_operation = FunctionType(
    i_o_operation.__code__,
    globals=globals(),
    closure=i_o_operation.__closure__[0].cell_contents.__closure__,
) 

Since i_o_operation is the result of inject(f) we can take that code to produce a new function. The globals is a formality that's required, and finally take the closure of the nested level, and the first part of the function is produced. Verify that the other is not called.

>>> i_operation('test', 'strip')
inject start
doing operation
inject finish
'teststrip'

Neat. However we still want the other to be wrapped outside of this to finally produce o_i_operation. We do need to somehow put this new function we produced in a closure, and a way to do this is to create a surrogate function that produce one

def closure(f):
    def surrogate(*a, **kw):
        return f(*a, **kw)
    return surrogate

And simply use it to construct and extract our closure

o_i_operation = FunctionType(
    i_o_operation.__closure__[0].cell_contents.__code__,
    globals=globals(),
    closure=closure(i_operation).__closure__,
)

Call this:

>>> o_i_operation('job', 'complete')
other start
inject start
doing operation
inject finish
other finish
'jobcomplete'

Looks like we finally got what we need. While this doesn't exactly answer your exact problem, this started down the right track but is already pretty hairy.

---

Now for the actual problem: a function that will ensure a decorator function be the most inner (final) callable before a given original, undecorated function - i.e. for a given target and a f(g(...(callable)), we want to emulate a result that gives f(g(...(target(callable)))). This is the code:

from types import FunctionType

def strip_decorators(f):
    """
    Strip all decorators from f.  Assumes each are functions with a
    closure with a first cell being the target function.
    """

    # list of not the actual decorator, but the returned functions
    decorators = []
    while f.__closure__:
        # Assume first item is the target method
        decorators.append(f)
        f = f.__closure__[0].cell_contents
    return decorators, f

def inject_decorator(decorator, f):
    """
    Inject a decorator to the most inner function within the stack of
    closures in `f`.
    """

    def closure(f):
        def surrogate(*a, **kw):
            return f(*a, **kw)
        return surrogate

    decorators, target_f = strip_decorators(f)
    result = decorator(target_f)

    while decorators:
        # pop out the last one in
        decorator = decorators.pop()
        result = FunctionType(
            decorator.__code__,
            globals=globals(),
            closure=closure(result).__closure__,
        )

    return result

To test this, we use a typical example use-case - html tags.

def italics(f):
    def i(s):
        return '<i>' + f(s) + '</i>'
    return i

def bold(f):
    def b(s):
        return '<b>' + f(s) + '</b>'
    return b

def underline(f):
    def u(s):
        return '<u>' + f(s) + '</u>'
    return u

@italics
@bold
def hi(s):
    return s

Running the test.

>>> hi('hello')
'<i><b>hello</b></i>'

Our target is to inject the underline decorator (specifically the u(hi) callable) into the most inner closure. This can be done like so, with the function we have defined above:

>>> hi_u = inject_decorator(underline, hi)
>>> hi_u('hello')
'<i><b><u>hello</u></b></i>'

Works with undecorated functions:

>>> def pp(s):
...     return s 
... 
>>> pp_b = inject_decorator(bold, pp)
>>> pp_b('hello')
'<b>hello</b>'

A major assumption was made for this first-cut version of the rewriter, which is that all decorators in the chain only have a closure length of one, that one element being the function being decorated with. Take this decorator for instance:

def prefix(p):
    def decorator(f):
        def inner(*args, **kwargs):
            new_args = [p + a for a in args]
            return f(*new_args, **kwargs)
        return inner
    return decorator

Example usage:

>>> @prefix('++')
... def prefix_hi(s):
...     return s
... 
>>> prefix_hi('test')
'++test'

Now try to inject a bold decorator like so:

>>> prefix_hi_bold = inject_decorator(bold, prefix_hi)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 18, in inject_decorator
ValueError: inner requires closure of length 2, not 1

This is simply because the closure formed by decorator within prefix has two elements, one being the prefix string p and the second being the actual function, and inner being nested inside that expects both those to be present inside its closure. Resolving that will require more code to analyse and reconstruct the details.

---

Anyway, this explanation took quite a bit of time and words, so I hope you understand this and maybe get you started on the actual right track.

If you want to turn inject_decorator into a decorator, and/or mix it into your class decorator, best of luck, most of the hard work is already done.