While troubleshooting a semi-related problem in the Python chat, I came upon some behavior in mypy that I don't understand.
from typing import Union, List, Dict
def f(x: Union[
Dict[str, float],
Dict[str, str],
Dict[str, int],
]):
pass
f({"a": 1}) #passes
f({"a": "b"}) #passes
f({"a": 1.0}) #passes
def g(x: Union[
Dict[str, float],
Dict[str, Union[str, int]],
]):
pass
g({"a": 1}) #fails
g({"a": "b"}) #fails
g({"a": 1.0}) #passes
def h(x: Dict[str, Union[float, str, int]]):
pass
h({"a": 1}) #passes
h({"a": "b"}) #passes
h({"a": 1.0}) #passes
When I execute mypy on this script, it only complains about the middle function, g
:
C:\Users\Kevin\Desktop>mypy test.py
test.py:20: error: Argument 1 to "g" has incompatible type "Dict[str, int]"; expected "Union[Dict[str, float], Dict[str, Union[str, int]]]"
test.py:20: note: "Dict" is invariant -- see http://mypy.readthedocs.io/en/latest/common_issues.html#variance
test.py:20: note: Consider using "Mapping" instead, which is covariant in the value type
test.py:21: error: Argument 1 to "g" has incompatible type "Dict[str, str]"; expected "Union[Dict[str, float], Dict[str, Union[str, int]]]"
test.py:21: note: "Dict" is invariant -- see http://mypy.readthedocs.io/en/latest/common_issues.html#variance
test.py:21: note: Consider using "Mapping" instead, which is covariant in the value type
Found 2 errors in 1 file (checked 1 source file)
(As the notes imply, replacing Dict
with Mapping
removes the errors, but let's say for the sake of the question that I must use Dict.)
These errors are surprising to me. As far as I can tell, the type annotations for each function should simplify down to the same group of types: a dict whose keys are strings, and whose values are floats/strings/ints. So why does only g
have incompatible types? Is mypy somehow confused by the presence of two Unions?
This is because Dict
is invariant. It should be invariant because it is mutable.
Dict[str, int]
is not a subtype of Dict[str, Union[str, int]]
(even though int
is a subtype of Union[int, str]
)
What if you are going to do something like this:
d: Dict[str, Union[str, int]]
u: Dict[str, int]
d = u # Mypy error: Incompatible type
d["Key"] = "value"
Mypy assumes that dictionaries are homogeneous: they will only ever contain one kind of type. In contrast to this, for example, Tuples
are meant to contain heterogeneous data: each item is allowed to have a different type.
If you need heterogenous Dict
, you could use TypedDict
, but only a fixed set of string keys is expected:
from typing import List, TypedDict
Mytype = TypedDict('Mytype', {'x': str, 'a': List[str]})
s: Mytype = {"x": "y", "a": ["b"]}
s['a'].append('c')
NOTE:
Unless you are on Python 3.8 or newer (where
TypedDict
is available in standard library typing module) you need to install typing_extensions using pip to use TypedDict
The issue is that Union
membership is modelled as subtyping, yet Dict
keys/values require exact type matches. This is enforced by MyPy for nested types (g
), but loosely interpreted for direct substitution ( h
).
The Union
type is modelled as a (virtual) subtyping relation. That is, str
is considered a subtype of Union[str, x]
.
issubbclass(a, (str, int))
checks whether a
is "either str
or int
" without saying which. We can use a str
value in place of a (str, int)
-union as long as only the common features are used.The Dict
type is invariant in its key/value types. That is, a key/value must be exactly of the type as declared.
d[k] = v
and output v = d[k]
. Any substitute would have to work with less capable input or provide more capable output – which is not possible, since input and output must have the same capabilities.In combination, using a Dict[..., Union[str, ...]]
requires the value to match the Union[str, ...]
exactly – the (virtual) subtype str
is not valid. As such, {"a": "b"}
is considered as Dict[str, str]
which is not a substitute for Dict[str, Union[str, int]]
.
Since logically a union follows slightly separate behaviour from regular types, MyPy has some separate code paths for Union
. This is mostly focused on function signatures and overloading, for which separate union math exists. Thus, certain flat type substitutions allows for more practical Union
matching, as is the case for h
.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With