Assuming I want to write a function that accepts any type of number in Python, I can annotate it as follows:
from numbers import Number
def foo(bar: Number):
print(bar)
Taking this concept one step further, I am writing functions which accept number types, i.e. int
, float
or numpy
dtypes, as arguments. Currently, I am writing:
from typing import Type
def foo(bar: Type):
assert issubclass(bar, Number)
print(bar)
I thought I could substitute Type
with something like NumberType
(similar to NotImplementedType
and friends, re-introduced in Python 3.10), because all number types are subclasses of Number
:
from numbers import Number
import numpy as np
assert issubclass(int, Number)
assert issubclass(np.uint8, Number)
As it turns out (or at least as far as I can tell), there is no such thing as a generic NumberType
in Python (3.9):
>>> type(Number)
abc.ABCMeta
Is there a clean way (i.e. without runtime checks) to achieve the desired kind of annotation?
Learn about classification of numbers. Identify the many types of numbers, including whole, natural, rational, irrational, even, odd, prime, composite, and integers. Updated: 10/30/2021 There are a finite number of numbers that we use in both mathematics and real-world situations.
To properly use real numbers, a system identifies the properties of each type of number. This classification system is called the real number system. The real number system is a system that separates all numbers into five categories. The image below shows the five groups within the real number system and how they relate to each other.
Numbers are strings of digits used to represent a quantity. The magnitude of a number indicates the size of the quantity. It can be either large or small. They exist in different forms, such as 3, 999, 0.351, 2/5, etc. Just like different family members live in different homes, different numbers are of the same family but have different types.
Natural numbers or counting numbers are the most basic types of numbers you learned for the first time as toddlers. They start from 1 and go to infinity, i.e., 1, 2, 3, 4, 5, 6, and so on. They are also called positive integers. In the set form, they can be written as:
In general, if we want to tell a type-checker that any instance of a certain class (or any instance of a subclass of that class) should be accepted as an argument to a function, we do it like so:
def accepts_int_instances(x: int) -> None:
pass
class IntSubclass(int):
pass
accepts_int_instances(42) # passes MyPy (an instance of `int`)
accepts_int_instances(IntSubclass(666)) # passes MyPy (an instance of a subclass of `int`)
accepts_int_instances(3.14) # fails MyPy (an instance of `float` — `float` is not a subclass of `int`)
Try it on MyPy playground here!
If, on the other hand, we have a class C
, and we want to hint that the class C
itself (or a subclass of C
) should be passed as an argument to a function, we use type[C]
rather than C
. (In Python <= 3.8, you will need to use typing.Type
instead of the builtin type
function, but as of Python 3.9 and PEP 585, we can parameterise type
directly.)
def accepts_int_and_subclasses(x: type[int]) -> None:
pass
class IntSubclass(int):
pass
accepts_int_and_subclasses(int) # passes MyPy
accepts_int_and_subclasses(float) # fails Mypy (not a subclass of `int`)
accepts_int_and_subclasses(IntSubclass) # passes MyPy
- Try it on MyPy playground here!
- Read the MyPy docs on "the type of class objects".
- See also: Subclass in type hinting.
How can we annotate a function to say that any numeric class should be accepted for a certain parameter?
int
, float
, and all the numpy
numeric types are all subclasses of numbers.Number
, so we should be able to just use type[Number]
if we want to say that all numeric classes are permitted!
At least, Python says that float
and int
are subclasses of Number
:
>>> from numbers import Number
>>> issubclass(int, Number)
True
>>> issubclass(float, Number)
True
And if we're using a runtime type-checking library such as typeguard, using type[Number]
seems to work fine:
>>> from typeguard import typechecked
>>> from fractions import Fraction
>>> from decimal import Decimal
>>> import numpy as np
>>>
>>> @typechecked
... def foo(bar: type[Number]) -> None:
... pass
...
>>> foo(str)
Traceback (most recent call last):
TypeError: argument "bar" must be a subclass of numbers.Number; got str instead
>>> foo(int)
>>> foo(float)
>>> foo(complex)
>>> foo(Decimal)
>>> foo(Fraction)
>>> foo(np.int64)
>>> foo(np.float32)
>>> foo(np.ulonglong)
>>> # etc.
But wait! If we try using type[Number]
with a static type-checker, it doesn't seem to work. If we run the following snippet through MyPy, it raises an error for each class except for fractions.Fraction
:
from numbers import Number
from fractions import Fraction
from decimal import Decimal
NumberType = type[Number]
def foo(bar: NumberType) -> None:
pass
foo(float) # fails
foo(int) # fails
foo(Fraction) # succeeds!
foo(Decimal) # fails
Try it on MyPy playground here!
Surely Python wouldn't lie to us about float
and int
being subclasses of Number
. What's going on?
type[Number]
doesn't work as a static type hint for numeric classesWhile issubclass(float, Number)
and issubclass(int, Number)
both evaluate to True
, neither float
nor int
is, in fact, a "strict" subclass of numbers.Number
. numbers.Number
is an Abstract Base Class, and int
and float
are both registered as "virtual subclasses" of Number
. This causes Python at runtime to recognise float
and int
as "subclasses" of Number
, even though Number
is not in the method resolution order of either of them.
See this StackOverflow question for an explanation of what a class's "method resolution order", or "mro", is.
>>> # All classes have `object` in their mro
>>> class Foo: pass
>>> Foo.__mro__
(<class '__main__.Foo'>, <class 'object'>)
>>>
>>> # Subclasses of a class have that class in their mro
>>> class IntSubclass(int): pass
>>> IntSubclass.__mro__
(<class '__main__.IntSubclass'>, <class 'int'>, <class 'object'>)
>>> issubclass(IntSubclass, int)
True
>>>
>>> # But `Number` is not in the mro of `int`...
>>> int.__mro__
(<class 'int'>, <class 'object'>)
>>> # ...Yet `int` still pretends to be a subclass of `Number`!
>>> from numbers import Number
>>> issubclass(int, Number)
True
>>> #?!?!!??
What's an Abstract Base Class? Why is
numbers.Number
an Abstract Base Class? What's "virtual subclassing"?
- The docs for Abstract Base Classes ("ABCs") are here.
- PEP 3119, introducing Abstract Base Classes, is here.
- The docs for the
numbers
module are here.- PEP 3141, which introduced
numbers.Number
, is here.- I can recommend this talk by Raymond Hettinger, which has a detailed explanation of ABCs and the purposes of virtual subclassing.
The issue is that MyPy does not understand the "virtual subclassing" mechanism that ABCs use (and, perhaps never will).
MyPy does understand some ABCs in the standard library. For instance, MyPy knows that list
is a subtype of collections.abc.MutableSequence
, even though MutableSequence
is an ABC, and list
is only a virtual subclass of MutableSequence
. However, MyPy only understands list
to be a subtype of MutableSequence
because we've been lying to MyPy about the method resolution order for list
.
MyPy, along with all other major type-checkers, uses the stubs found in the typeshed repository for its static analysis of the classes and modules found in the standard library. If you look at the stub for list
in typeshed, you'll see that list
is given as being a direct subclass of collections.abc.MutableSequence
. That's not true at all — MutableSequence
is written in pure Python, whereas list
is an optimised data structure written in C. But for static analysis, it's useful for MyPy to think that this is true. Other collections classes in the standard library (for example, tuple
, set
, and dict
) are special-cased by typeshed in much the same way, but numeric types such as int
and float
are not.
If we lie to MyPy about collections classes, why don't we also lie to MyPy about numeric classes?
A lot of people (including me!) think we should, and discussions have been ongoing for a long time about whether this change should be made (e.g. typeshed proposal, MyPy issue). However, there are various complications to doing so.
Credit goes to @chepner in the comments for finding the link to the MyPy issue.
One possible (albeit slightly icky) solution here might be to use typing.SupportsFloat
.
SupportsFloat
is a runtime-checkable protocol that has a single abstractmethod
, __float__
. This means that any classes that have a __float__
method are recognised — both at runtime and by static type-checkers — as being subtypes of SupportsFloat
, even if SupportsFloat
is not in the class's method resolution order.
What's a protocol? What's duck-typing? How do protocols do what they do? Why are some protocols, but not all protocols, checkable at runtime?
- PEP 544, introducing
typing.Protocol
and structural-typing/duck-typing, explains in detail howtyping.Protocol
works. It also explains how static type-checkers are able to recognise classes such asfloat
andint
as subtypes ofSupportsFloat
, even thoughSupportsFloat
does not appear in the method resolution order forint
orfloat
.- The Python docs for structural subtyping are here.
- The Python docs for
typing.Protocol
are here.- The MyPy docs for
typing.Protocol
are here.- The Python docs for
typing.SupportsFloat
are here.- The source code for
typing.SupportsFloat
is here.- By default, protocols cannot be checked at runtime with
isinstance
andissubclass
.SupportsFloat
is checkable at runtime because it is decorated with the@runtime_checkable
decorator. Read the documentation for that decorator here.
Note: although user-defined protocols are only available in Python >= 3.8, SupportsFloat
has been in the typing
module since the module's addition to the standard library in Python 3.5.
Advantages of this solution
Comprehensive support* for all major numeric types: fractions.Fraction
, decimal.Decimal
, int
, float
, np.int32
, np.int16
, np.int8
, np.int64
, np.int0
, np.float16
, np.float32
, np.float64
, np.float128
, np.intc
, np.uintc
, np.int_
, np.uint
, np.longlong
, np.ulonglong
, np.half
, np.single
, np.double
, np.longdouble
, np.csingle
, np.cdouble
, and np.clongdouble
all have a __float__
method.
If we annotate a function argument as being of type[SupportsFloat]
, MyPy correctly accepts* the types that conform to the protocol, and correctly rejects the types that do not conform to the protocol.
It is a fairly general solution — you do not need to explicitly enumerate all possible types that you wish to accept.
Works with both static type-checkers and runtime type-checking libraries such as typeguard
.
Disadvantages of this solution
It feels like (and is) a hack. Having a __float__
method isn't anybody's reasonable idea of what defines a "number" in the abstract.
Mypy does not recognise complex
as a subtype of SupportsFloat
. complex
does, in fact, have a __float__
method in Python <= 3.9. However, it does not have a __float__
method in the typeshed stub for complex
. Since MyPy (along with all other major type-checkers) uses typeshed stubs for its static analysis, this means it is unaware that complex
has this method. complex.__float__
is likely omitted from the typeshed stub due to the fact that the method always raises TypeError
; for this reason, the __float__
method has in fact been removed from the complex
class in Python 3.10.
Any user-defined class, even if it is not a numeric class, could potentially define __float__
. In fact, there are even several non-numeric classes in the standard library that define __float__
. For example, although the str
type in Python (which is written in C) does not have a __float__
method, collections.UserString
(which is written in pure Python) does. (The source code for str
is here, and the source code for collections.UserString
is here.)
Example usage
This passes MyPy for all numeric types I tested it with, except for complex
:
from typing import SupportsFloat
NumberType = type[SupportsFloat]
def foo(bar: NumberType) -> None:
pass
Try it on MyPy playground here!
If we want complex
to be accepted as well, a naive tweak to this solution would just be to use the following snippet instead, special-casing complex
. This satisfies MyPy for every numeric type I could think of. I've also thrown type[Number]
into the type hint, as it could be useful to catch a hypothetical class that does directly inherit from numbers.Number
and does not have a __float__
method. I don't know why anybody would write such a class, but there are some classes that directly inherit from numbers.Number
(e.g. fractions.Fraction
), and it certainly would be theoretically possible to create a direct subclass of Number
without a __float__
method. Number
itself is an empty class that has no methods — it exists solely to provide a "virtual base class" for other numeric classes in the standard library.
from typing import SupportsFloat, Union
from numbers import Number
NumberType = Union[type[SupportsFloat], type[complex], type[Number]]
# You can also write this more succinctly as:
# NumberType = type[Union[SupportsFloat, complex, Number]]
# The two are equivalent.
# In Python >= 3.10, we can even write it like this:
# NumberType = type[SupportsFloat | complex | Number]
# See PEP 604: https://www.python.org/dev/peps/pep-0604/
def foo(bar: NumberType) -> None:
pass
Try it on MyPy playground here!
Translated into English, NumberType
here is equivalent to:
Any class, if and only if:
- It has a
__float__
method;- AND/OR it is
complex
;- AND/OR it is a subclass of
complex
;- AND/OR it is
numbers.Number
;- AND/OR it is a "strict" (non-virtual) subclass of
numbers.Number
.
I don't see this as a "solution" to the problem with complex
— it's more of a workaround. The issue with complex
is illustrative of the dangers of this approach in general. There may be other unusual numeric types in third-party libraries, for example, that do not directly subclass numbers.Number
or have a __float__
method. It would be exceedingly difficult to know what they might look like in advance, and special-case them all.
Why SupportsFloat
instead of typing.SupportsInt
?
fractions.Fraction
has a __float__
method (inherited from numbers.Rational
) but does not have an __int__
method.
Why SupportsFloat
instead of SupportsAbs
?
Even complex
has an __abs__
method, so typing.SupportsAbs
looks like a promising alternative at first glance! However, there are several other classes in the standard library that have an __abs__
method and do not have a __float__
method, and it would be a stretch to argue that they are all numeric classes. (datetime.timedelta
doesn't feel very number-like to me.) If you used SupportsAbs
rather than SupportsFloat
, you would risk casting your net too wide and allowing all sorts of non-numeric classes.
Why SupportsFloat
instead of SupportsRound
?
As an alternative to SupportsFloat
, you could also consider using typing.SupportsRound
, which accepts all classes that have a __round__
method. This is just as comprehensive as SupportsFloat
(it covers all major numeric types other than complex
). It also has the advantage that collection.UserString
has no __round__
method whereas, as discussed above, it does have a __float__
method. Lastly, it seems less likely that third-party non-numeric classes would include a __round__
method.
However, if you went for SupportsRound
rather than SupportsFloat
, you would, in my opinion, run a greater risk of excluding valid third-party numeric classes that, for whatever reason, do not define __round__
.
"Having a __float__
method" and "having a __round__
method" are both pretty poor definitions of what it means for a class to be a "number". However, the former feels much closer to the "true" definition than the latter. As such, it feels safer to count on third-party numeric classes having a __float__
method than it does to count on them having a __round__
method.
If you wanted to be "extra safe" when it comes to ensuring a valid third-party numeric type is accepted by your function, I can't see any particular harm in extending NumberType
even further with SupportsRound
:
from typing import SupportsFloat, SupportsRound, Union
from numbers import Number
NumberType = Union[type[SupportsFloat], type[SupportsRound], type[complex], type[Number]]
However, I would question whether it's really necessary to include SupportsRound
, given that any type that has a __round__
method is very likely to have a __float__
method as well.
*...except for complex
There is no general way to do this. Numbers are not strictly related to begin with and their types are even less.
While numbers.Number
might seem like "the type of numbers" it is not universal. For example, decimal.Decimal
is explicitly not a numbers.Number
as either subclass, subtype or virtual subclass. Specifically for typing, numbers.Number
is not endorsed by PEP 484 -- Type Hints.
In order to meaningfully type hint "numbers", one has to explicitly define what numbers are in that context. This might be a pre-existing numeric type set such as int
<: float
<: complex
, a typing.Union
/TypeVar
of numeric types, a typing.Protocol
to define operations and algebraic structure, or similar.
from typing import TypeVar
from decimal import Decimal
from fractions import Fraction
#: typevar of rational numbers if we squint real hard
Q = TypeVar("Q", float, Decimal, Fraction)
All that said, "the type of the type of numbers" is even less meaningful. Even the specific numbers.Number
has practically no features at all: it cannot be converted to a concrete type, nor can it be instantiated to a meaningful number.
Instead, use "the type of some type of numbers":
from typing import Type
def zero(t: Type[Q]) -> Q:
return t() # all Type[Q]s can be instantiated without arguments
print(zero(Fraction))
If the only goal of the Type
is to create instances, it can be better to request a Callable
instead. This covers types as well as factory functions.
def one(t: Callable[[int], Q]) -> Q:
return t(1)
This is not quite an answer to the original question. (Alex Waygood's answer is properly selected as responsive.) However, I have attempted to generalize my own work-arounds for the sharp edges between numbers and typing in Python. Those work-arounds now live in numerary
(having extracted it via c-section from dyce
, where it was conceived).
I didn't spend a lot of time on naming, in the hopes it will be short-lived. Docs are online. It should be considered experimental, but it is rapidly approaching stability. Feedback, suggestions, and contributions are desperately appreciated.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With