Is there any benefit to using Py_DECREF instead of Py_XDECREF for Python C Extensions?

Question

I'm working through the Python C Extension documentation for defining new types and just finished the Providing finer control over data attributes section.

In this section, they change the example code to ensure that the first and last attributes of the Noddy struct can never be NULL, for example by initializing the attributes to empty strings in new and adding getters and setters that raise TypeError if the user tries to delete or otherwise set these attributes to Null.

Additionally (and the point of my question), the author changes all of the Py_XDECREF to Py_DECREF for these attributes, stating that:

With these changes, we can assure that the first and last members are never NULL so we can remove checks for NULL values in almost all cases. This means that most of the Py_XDECREF() calls can be converted to Py_DECREF() calls. The only place we can’t change these calls is in the deallocator, where there is the possibility that the initialization of these members failed in the constructor.

It seems to me that it would just be safer to use Py_XDECREF, given that Py_DECREF results in a segmentation fault if its passed a NULL value.

What is the benefit to using Py_DECREF over Py_XDECREF?

Answer 1

If you know that the object cannot be NULL, the benefits of Py_DECREF over Py_XDECREF are that:

• it is faster, since it avoids an unnecessary test and jump;
• it signals to the (human) reader that the pointer is expected to be non-NULL at the point of deallocation;
• it provides what is effectively a zero-cost assertion that the pointer is non-NULL - the program is likely to immediately crash if the invariant is broken.¹

These points can be important when dealing with low-level code, which is why Python core and most extensions are careful to only use Py_XDECREF (or Py_CLEAR in tp_clear) when the pointer can actually be NULL.

¹ Technically it's undefined behavior, which means that a crash is not guaranteed the way it would be with an actual assertion. In practice, however, the compiler has little choice but to generate code that dereferences the pointer, which will lead to a memory fault if it is NULL.