Type annotation for Callable that takes **kwargs

Question

There is a function (f) which consumes a function signature (g) that takes a known first set of arguments and any number of keyword arguments **kwargs. Is there a way to include the **kwargs in the type signature of (g) that is described in (f)?

For example:

from typing import Callable, Any
from functools import wraps
import math


def comparator(f: Callable[[Any, Any], bool]) -> Callable[[str], bool]:
    @wraps(f)
    def wrapper(input_string: str, **kwargs) -> bool:
        a, b, *_ = input_string.split(" ")
        return f(eval(a), eval(b), **kwargs)

    return wrapper


@comparator
def equal(a, b):
    return a == b


@comparator
def equal_within(a, b, rel_tol=1e-09, abs_tol=0.0):
    return math.isclose(a, b, rel_tol=rel_tol, abs_tol=abs_tol)


# All following statements should print `True`
print(equal("1 1") == True)
print(equal("1 2") == False)
print(equal_within("5.0 4.99998", rel_tol=1e-5) == True)
print(equal_within("5.0 4.99998") == False)

The function comparator wraps its argument f with wrapper, which consumes the input for f as a string, parses it and evaluates it using f. In this case, Pycharm gives a warning that return f(eval(a), eval(b), **kwargs) calls f with the unexpected argument **kwargs, which doesn't match the expected signature.

This post on Reddit suggests adding either Any or ... to the type signature of f like

• f: Callable[[Any, Any, ...], bool]
• f: Callable[[Any, Any, Any], bool]

The former causes a TypeError [1], while the latter seems to misleading, since f accepts at least 2 arguments, rather than exactly 3.

Another workaround is to leave the Callable args definition open with ... like f: Callable[..., bool], but I'm wondering if there is a more appropriate solution.

1. TypeError: Callable[[arg, ...], result]: each arg must be a type. Got Ellipsis.

Answer 1

tl;dr: Protocol may be the closest feature that's implemented, but it's still not sufficient for what you need. See this issue for details.

---

Full answer:

I think the closest feature to what you're asking for is Protocol, which was introduced in Python 3.8 (and backported to older Pythons via typing_extensions). It allows you to define a Protocol subclass that describes the behaviors of the type, pretty much like an "interface" or "trait" in other languages. For functions, a similar syntax is supported:

from typing import Protocol
# from typing_extensions import Protocol  # if you're using Python 3.6

class MyFunction(Protocol):
    def __call__(self, a: Any, b: Any, **kwargs) -> bool: ...

def decorator(func: MyFunction):
    ...

@decorator  # this type-checks
def my_function(a, b, **kwargs) -> bool:
    return a == b

In this case, any function that have a matching signature can match the MyFunction type.

However, this is not sufficient for your requirements. In order for the function signatures to match, the function must be able to accept an arbitrary number of keyword arguments (i.e., have a **kwargs argument). To this point, there's still no way of specifying that the function may (optionally) take any keyword arguments. This GitHub issue discusses some possible (albeit verbose or complicated) solutions under the current restrictions.

---

For now, I would suggest just using Callable[..., bool] as the type annotation for f. It is possible, though, to use Protocol to refine the return type of the wrapper:

class ReturnFunc(Protocol):
    def __call__(self, s: str, **kwargs) -> bool: ...

def comparator(f: Callable[..., bool]) -> ReturnFunc:
    ....

This gets rid of the "unexpected keyword argument" error at equal_within("5.0 4.99998", rel_tol=1e-5).

Answer 2

With PEP 612 in Python 3.10, you may try the following solution:

from typing import Callable, Any, ParamSpec, Concatenate
from functools import wraps

P = ParamSpec("P")


def comparator(f: Callable[Concatenate[Any, Any, P], bool]) -> Callable[Concatenate[str, P], bool]:
    @wraps(f)
    def wrapper(input_string: str, *args: P.args, **kwargs: P.kwargs) -> bool:
        a, b, *_ = input_string.split(" ")
        return f(eval(a), eval(b), *args, **kwargs)

    return wrapper

However it seems that you cannot get rid of *args: P.args (which you actually don't need) as PEP 612 requires P.args and P.kwargs to be used together. If you can make sure that your decorated functions (e.g. equal and equal_within) do not take extra positional arguments, any attempts to call the functions with extra positional arguments should be rejected by the type checker.