Class invariants in Python

Question

Class invariants definitely can be useful in coding, as they can give instant feedback when clear programming error has been detected and also they improve code readability as being explicit about what arguments and return value can be. I'm sure this applies to Python too.

However, generally in Python, testing of arguments seems not to be "pythonic" way to do things, as it is against the duck-typing idiom.

My questions are:

1. What is Pythonic way to use assertions in code?

   For example, if I had following function:

   def do_something(name, path, client):
       assert isinstance(name, str)
       assert path.endswith('/')
       assert hasattr(client, "connect")
   

2. More generally, when there is too much of assertions?

I'd be happy to hear your opinions on this!

Answer 1

Short Answer:

Are assertions Pythonic?

Depends how you use them. Generally, no. Making generalized, flexible code is the most Pythonic thing to do, but when you need to check invariants:

1. Use type hinting to help your IDE perform type inference so you can avoid potential pitfalls.

2. Make robust unit tests.

3. Prefer try/except clauses that raise more specific exceptions.

4. Turn attributes into properties so you can control their getters and setters.

5. Use assert statements only for debug purposes.

Refer to this Stack Overflow discussion for more info on best practices.

Long Answer

You're right. It's not considered Pythonic to have strict class invariants, but there is a built-in way to designate the preferred types of parameters and returns called type hinting, as defined in PEP 484:

[Type hinting] aims to provide a standard syntax for type annotations, opening up Python code to easier static analysis and refactoring, potential runtime type checking, and (perhaps, in some contexts) code generation utilizing type information.

The format is this:

def greeting(name: str) -> str:
    return 'Hello ' + name 

The typing library provides even further functionality. However, there's a huge caveat...

While these annotations are available at runtime through the usual __annotations__ attribute, no type checking happens at runtime . Instead, the proposal assumes the existence of a separate off-line type checker which users can run over their source code voluntarily. Essentially, such a type checker acts as a very powerful linter.

Whoops. Well, you could use an external tool while testing to check when invariance is broken, but that doesn't really answer your question.

---

Properties and try/except

The best way to handle an error is to make sure it never happens in the first place. The second best way is to have a plan when it does. Take, for example, a class like this:

 class Dog(object):
     """Canis lupus familiaris."""

     self.name = str()
     """The name you call it."""


     def __init__(self, name: str):
         """What're you gonna name him?"""

         self.name = name


     def speak(self, repeat=0):
         """Make dog bark. Can optionally be repeated."""

         print("{dog} stares at you blankly.".format(dog=self.name))

         for i in range(repeat):
             print("{dog} says: 'Woof!'".format(dog=self.name)

If you want your dog's name to be an invariant, this won't actually prevent self.name from being overwritten. It also doesn't prevent parameters that could crash speak(). However, if you make self.name a property...

 class Dog(object):
     """Canis lupus familiaris."""
     
     self._name = str()
     """The name on the microchip."""

     self.name = property()
     """The name on the collar."""


     def __init__(self, name: str):
         """What're you gonna name him?"""

         if not name and not name.isalpha():
             raise ValueError("Name must exist and be pronouncable.")

         self._name = name


     def speak(self, repeat=0):
         """Make dog bark. Can optionally be repeated."""

         try:
             print("{dog} stares at you blankly".format(dog=self.name))
             
             if repeat < 0:
                 raise ValueError("Cannot negatively bark.")

             for i in range(repeat):
                 print("{dog} says: 'Woof!'".format(dog=self.name))

         except (ValueError, TypeError) as e:
             raise RuntimeError("Dog unable to speak.") from e


     @property
     def name(self):
         """Gets name."""

         return self._name

Since our property doesn't have a setter, self.name is essentially invariant; that value can't change unless someone is aware of the self._x. Furthermore, since we've added try/except clauses to process the specific errors we're expecting, we've provided a more concise control flow for our program.

---

So When Do You Use Assertions?

There might not be a 100% "Pythonic" way to perform assertions since you should be doing those in your unit tests. However, if it's critical at runtime for data to be invariant, assert statements can be used to pinpoint possible trouble spots, as explained in the Python wiki:

Assertions are particularly useful in Python because of Python's powerful and flexible dynamic typing system. In the same example, we might want to make sure that ids are always numeric: this will protect against internal bugs, and also against the likely case of somebody getting confused and calling by_name when they meant by_id.

For example:

from types import *
  class MyDB:
  ...
  def add(self, id, name):
    assert type(id) is IntType, "id is not an integer: %r" % id
    assert type(name) is StringType, "name is not a string: %r" % name

Note that the "types" module is explicitly "safe for import *"; everything it exports ends in "Type".

That takes care of data type checking. For classes, you use isinstance(), as you did in your example:

You can also do this for classes, but the syntax is a little different:

class PrintQueueList:
  ...
  def add(self, new_queue):
   assert new_queue not in self._list, \
     "%r is already in %r" % (self, new_queue)
   assert isinstance(new_queue, PrintQueue), \
     "%r is not a print queue" % new_queue

I realize that's not the exact way our function works but you get the idea: we want to protect against being called incorrectly. You can also see how printing the string representation of the objects involved in the error will help with debugging.

For proper form, attaching a message to your assertions like in the examples above
(ex: assert <statement>, "<message>") will automatically attach the info into the resulting AssertionError to assist you with debugging. It could also give some insight into a consumer bug report as to why the program is crashing.

Checking isinstance() should not be overused: if it quacks like a duck, there's perhaps no need to enquire too deeply into whether it really is. Sometimes it can be useful to pass values that were not anticipated by the original programmer.

Places to consider putting assertions:

• checking parameter types, classes, or values
• checking data structure invariants
• checking "can't happen" situations (duplicates in a list, contradictory state variables.)
• after calling a function, to make sure that its return is reasonable

Assertions can be beneficial if they're properly used, but you shouldn't become dependent on them for data that doesn't need to be explicitly invariant. You might need to refactor your code if you want it to be more Pythonic.

Answer 2

Please have a look at icontract library. We developed it to bring design-by-contract into Python with informative error messages. Here as an example of a class invariant:

>>> @icontract.inv(lambda self: self.x > 0)
... class SomeClass:
...     def __init__(self) -> None:
...         self.x = 100
...
...     def some_method(self) -> None:
...         self.x = -1
...
...     def __repr__(self) -> str:
...         return "some instance"
...
>>> some_instance = SomeClass()
>>> some_instance.some_method()
Traceback (most recent call last):
 ...
icontract.ViolationError: self.x > 0:
self was some instance
self.x was -1