What's the idiomatic way to increment a Python new integer type?

Question

Let's say I define a new type as such:

import typing

Index = typing.NewType('Index', int)

Then let's say I have an Index variable as such:

index = Index(0)

What would be the idiomatic way to increment index?

If I do

index += 1

which is equivalent to

index = index + 1

then the type of index becomes int instead of Index from a static type checking point of view:

$ mypy example.py

example.py:4: error: Incompatible types in assignment (expression has type "int", variable has type "Index")

Is there anything better than

index = Index(index + 1)

to keep the Index type?

Answer 1

You have to create an int subclass "for real" (pun not intended, but it is bad enough to stay there).

There are two problems there:

• typing.NewType does not create a new class. It just separate a "lineage" of objects so that they will "look like" a new class to static-type checking tools - but objects created with such a class are still of the indicated class in runtime.

See "typing.Newtype" at work in the interactive prompt, rather than relying on the static-check report:

In [31]: import typing                                                                                                                                                              

In [32]: Index = typing.NewType("Index", int)                                                                                                                                       

In [33]: a = Index(5)                                                                                                                                                               

In [34]: type(a)                                                                                                                                                                    
Out[34]: int

• The second problem, is that even if you subclass int the proper way, the operations that result from applying any operator will still cast the result type back to int, and won't be of the created subclass:

In [35]: class Index(int): pass                                                                                                                                                     

In [36]: a = Index(5)                                                                                                                                                               

In [37]: type(a)                                                                                                                                                                    
Out[37]: __main__.Index

In [38]: type(a + 1)                                                                                                                                                                
Out[38]: int

In [39]: type(a + a)                                                                                                                                                                
Out[39]: int

In [40]: a += 1                                                                                                                                                                     

In [41]: type(a)                                                                                                                                                                    
Out[41]: int

So, the only way out is to actually wrap all the magic methods which perform numeric operations in functions that "cast" the result back to the subclass. One can avoid repeating the same pattern several times in the class body by creating a decorator to perform this casting, and appling it to all numeric methods in a for loop in the class body itself.

In [68]: num_meths = ['__abs__', '__add__', '__and__',  '__ceil__', 
'__divmod__',  '__floor__', '__floordiv__', '__invert__', '__lshift__',
'__mod__', '__mul__', '__neg__', '__pos__', '__pow__', '__radd__', 
'__rand__', '__rdivmod__',  '__rfloordiv__', '__rlshift__', '__rmod__',
'__rmul__', '__ror__', '__round__', '__rpow__', '__rrshift__', 
'__rshift__', '__rsub__', '__rtruediv__', '__rxor__', '__sub__', 
'__truediv__', '__trunc__', '__xor__',  'from_bytes'] 
# to get these, I did `print(dir(int))` copy-pasted the result, and 
# deleted all non-relevant methods to this case, leaving only the ones 
# that perform operations which should preserve the subclass


In [70]: class Index(int): 
             for meth in num_meths: 
                locals()[meth] = (lambda meth:
                    lambda self, *args:
                         __class__(getattr(int, meth)(self, *args))
                    )(meth) 
# creating another "lambda" layer to pass the value of meth in _each_
# iteration of the for loop is needed so that these values are "frozen" 
# for each created method

In [71]: a = Index(5)                                                                                                                                                               

In [72]: type(a)                                                                                                                                                                    
Out[72]: __main__.Index

In [73]: type(a + 1)                                                                                                                                                                
Out[73]: __main__.Index

In [74]: a += 1                                                                                                                                                                     

In [75]: type(a)                                                                                                                                                                    
Out[75]: __main__.Index

This will actually work.

However, if the intent is that static type checking "sees" this wrapping taking place, you are off-the trail once more. Static type checkers won't understand method creation by applying a decorator in a loop inside a class body.

In other words, I see no way out of this but by copy-and-pasting the casting applyed automatically in the example above in all relevant numeric methods, and then, create annotations while at it:

from __future__ import annotations
from typing import Union

class Index(int):
    def __add__(self: Index, other: Union[Index, int]) -> Index:
        return __class__(super().__add__(other))
    def __radd__(self: Index, other: Union[Index, int]) -> Index:
        return __class__(super().__radd__(other))
    # Rinse and repeat for all numeric methods you intend to use

Answer 2

I think that you will not be able to implicitly(indirectly) keep the type as a result of an operation on a NewType variable, because as the documentation says:

You may still perform all int operations on a variable of type UserId, but the result will always be of type int

Thus, any operation on a variable will produce a result of the base type. If you want the result to be a NewType, you need to specify it explicitly, e.g. as index = Index(index + 1). Here Index works as cast function.

This is consistent with the purpose of NewType:

The static type checker will treat the new type as if it were a subclass of the original type. This is useful in helping catch logical errors

The intended purpose of NewType is to help you detect cases where you accidentally mixed together the old base type and the new derived type.