getting sphinx to recognise correct signature

Question

I've been trying to get my documentation in order for an open source project I'm working on, which involves a mirrored client and server API. To this end I have created a decorator that can most of the time be used to document a method that simply performs validation on its input. You can find a class full of these methods here and the decorator's implementation here.

The decorator, as you can see uses functools.wraps to preserve the docstring, and I thought also the signature, however the source code vs the generated documentation looks like this:

Source:

vs

Docs:

Does anyone know any way to have setH's generated documentation show the correct call signature? (without having a new decorator for each signature - there are hudreds of methods I need to mirror)

I've found a workaround which involved having the decorator not changing the unbound method, but having the class mutate the method at binding time (object instantiation) - this seems like a hack though, so any comments on this, or alternative ways of doing this, would be appreciated.

Answer 1

I'd like to avoid reliance on too muck outside of the standard library, so while I have looked at the Decorator module, I have mainly tried to reproduce its functionality.... Unsuccessfully...

So I took a look at the problem from another angle, and now I have a partially working solution, which can mainly be described by just looking at this commit. It's not perfect as it relies on using partial, which clobbers the help in the REPL. The idea is that instead of replacing the function to which the decorator is applied, it is augmented with attributes.

+def s_repr(obj):
+    """ :param obj: object """
+    return (repr(obj) if not isinstance(obj, SikuliClass)
+            else "self._get_jython_object(%r)" % obj._str_get)
+
+
 def run_on_remote(func):
     ...
-    func.s_repr = lambda obj: (repr(obj)
-                               if not isinstance(obj, SikuliClass) else
-                               "self._get_jython_object(%r)" % obj._str_get)
-
-    def _inner(self, *args):
-        return self.remote._eval("self._get_jython_object(%r).%s(%s)" % (
-            self._id,
-            func.__name__,
-            ', '.join([func.s_repr(x) for x in args])))
-
-    func.func = _inner
+    gjo = "self._get_jython_object"
+    func._augment = {
+        'inner': lambda self, *args: (self.remote._eval("%s(%r).%s(%s)"
+                                      % (gjo, self._id, func.__name__,
+                                         ', '.join([s_repr(x)for x in args]))))
+    }

     @wraps(func)
     def _outer(self, *args, **kwargs):
         func(self, *args, **kwargs)
-        if hasattr(func, "arg"):
-            args, kwargs = func.arg(*args, **kwargs), {}
-        result = func.func(*args, **kwargs)
-        if hasattr(func, "post"):
+        if "arg" in func._augment:
+            args, kwargs = func._augment["arg"](self, *args, **kwargs), {}
+        result = func._augment['inner'](self, *args, **kwargs)
+        if "post" in func._augment:
             return func.post(result)
         else:
             return result

     def _arg(arg_func):
-        func.arg = arg_func
-        return _outer
+        func._augment['arg'] = arg_func
+        return func

     def _post(post_func):
-        func.post = post_func
-        return _outer
+        func._augment['post'] = post_func
+        return func

     def _func(func_func):
-        func.func = func_func
-        return _outer
-    _outer.arg = _arg
-    _outer.post = _post
-    _outer.func = _func
-    return _outer
+        func._augment['inner'] = func_func
+        return func
+
+    func.arg  = _outer.arg = _arg
+    func.post = _outer.post = _post
+    func.func = _outer.func = _func
+    func.run  = _outer.run = _outer
+    return func

So this doesn't actually change the unbound method, ergo the generated documentation stays the same. The second part of the trickery occurs at class initialisation.

 class ClientSikuliClass(ServerSikuliClass):
     """ Base class for types based on the Sikuli native types """
     ...
     def __init__(self, remote, server_id, *args, **kwargs):
         """
         :type server_id: int
         :type remote: SikuliClient
         """
         super(ClientSikuliClass, self).__init__(None)
+        for key in dir(self):
+            try:
+                func = getattr(self, key)
+            except AttributeError:
+                pass
+            else:
+                try:
+                    from functools import partial, wraps
+                    run = wraps(func.run)(partial(func.run, self))
+                    setattr(self, key, run)
+                except AttributeError:
+                    pass
         self.remote = remote
         self.server_id = server_id

So at the point where an instance of any class inheriting ClientSikuliClass is instantiated, an attempt is made to take the run property of each attribute of that instance and make that what is returned on attempting to get that attribute, and so the bound method is now a partially applied _outer function.

So the issues with this are multiple:

1. Using partial at initilaisation results in losing the bound method information.
2. I worry about clobbering attributes that just so happen to have a run attribute...

So while I have an answer to my own question, I'm not quite satisfied by it.

---

Update

Ok so after a bit more work I ended up with this:

 class ClientSikuliClass(ServerSikuliClass):
     """ Base class for types based on the Sikuli native types """
     ...
     def __init__(self, remote, server_id, *args, **kwargs):
         """
         :type server_id: int
         :type remote: SikuliClient
         """
         super(ClientSikuliClass, self).__init__(None)
-        for key in dir(self):
+
+        def _apply_key(key):
             try:
                 func = getattr(self, key)
+                aug = func._augment
+                runner = func.run
             except AttributeError:
-                pass
-            else:
-                try:
-                    from functools import partial, wraps
-                    run = wraps(func.run)(partial(func.run, self))
-                    setattr(self, key, run)
-                except AttributeError:
-                    pass
+                return
+
+            @wraps(func)
+            def _outer(*args, **kwargs):
+                return runner(self, *args, **kwargs)
+
+            setattr(self, key, _outer)
+
+        for key in dir(self):
+            _apply_key(key)
+
         self.remote = remote
         self.server_id = server_id

This prevents the loss of the documentation on the object. You'll also see that the func._augment attribute is accessed, even though it is not used, so that if it does not exist the object attribute will not be touched.

I'd be interested if anyone had any comments on this?

Answer 2

In PRAW, I handled this issue by having conditional decorators that return the original function (rather than the decorated function) when a sphinx build is occurring.

In PRAW's sphinx conf.py I added the following as a way to determine if SPHINX is currently building:

import os
os.environ['SPHINX_BUILD'] = '1'

And then in PRAW, its decorators look like:

import os

# Don't decorate functions when building the documentation
IS_SPHINX_BUILD = bool(os.getenv('SPHINX_BUILD'))

def limit_chars(function):
    """Truncate the string returned from a function and return the result."""
    @wraps(function)
    def wrapped(self, *args, **kwargs):
        output_string = function(self, *args, **kwargs)
        if len(output_string) > MAX_CHARS:
            output_string = output_string[:MAX_CHARS - 3] + '...'
        return output_string
    return function if IS_SPHINX_BUILD else wrapped

The return function if IS_SPHINX_BUILD else wrapped line is what allows SPHINX to pick up the correct signature.

Relevant Source