How can I make "isinstance" on Protocols also include function signatures and data-types?

Question

The following code defines a simple protocol and a class which almost implements that protocol. The only difference is the fact that the run() method takes an argument in the protocol, but it's not implemented that way.

However, the isinstance() check returns true which was unexpected.

As I understood PEP-544 this should work. Although it's pretty unclear about checking function signatures.

The same issue happens when using the wrong data-type in the ORDER member (f.ex. changing it to str in the protocol).

I'm aware that type-hints are just... well... "hints" and are not enforced at runtime.

However, in my application it would be useful to ensure certain classes follow a defined protocol for clearer error-messages. It is using a plugin-architecture, and after loading a plugin it would be useful to have a quick "sanity-check" if that plugin follows the required protocol and if not, give an early and useful error-message instead of causing an exception later on downstream.

from typing_extensions import Protocol, runtime_checkable


@runtime_checkable
class PFoo(Protocol):
    ORDER: int

    def run(self, a: int) -> None:
        ...


class Hello:
    ORDER = 10

    def run(self) -> None:
        print(1)


# Returns "True" evn though the signature of "run()" doesn't match
print(isinstance(Hello(), PFoo))

Answer 1

Here's the docstring for the runtime_checkable function in the typing module:

"""Mark a protocol class as a runtime protocol.

Such protocol can be used with isinstance() and issubclass().
Raise TypeError if applied to a non-protocol class.
This allows a simple-minded structural check very similar to
one trick ponies in collections.abc such as Iterable.
For example::

    @runtime_checkable
    class Closable(Protocol):
        def close(self): ...

    assert isinstance(open('/some/file'), Closable)

Warning: this will check only the presence of the required methods,
not their type signatures!
"""

If you want to compare type-annotations of functions, you can examine the __annotations__ attribute of the function (if it's a module that has from __future__ import annotations enabled, it may be better to use typing.get_type_hints on the function rather than examining the __annotations__ attribute directly). And if you're looking for an exact signature match, you can use the signature function from the inspect module — docs here. Beware: I think using inspect.signature may incur some runtime performance costs (but then again, so does all runtime type-checking).