Returning from __len__() when >64 bits

Question

In this problem, I'm dealing with IPv6 network address spaces, so the length is 2^(128-subnet).

It appears that python (at least on this machine), will cope with up to a 64 bit signed number as the return value from __len__(). So len(IP('2001::/66')) works, but len(IP('2001::/65')) fails.

from IPy import IP
len(IP('2001::/64'))
Traceback (most recent call last):
  File "test.py", line 2, in <module>
      len(IP('2001::/64'))
OverflowError: long int too large to convert to int

The IPy library in question is at https://github.com/haypo/python-ipy.

Any suggestions on how to handle this, or hint that it might be a limitation I'm stuck with?

Answer 1

The issue you're hitting is that Python's C API has a system-dependent limit on the lengths of containers. That is, the C function PyObject_Size returns a Py_ssize_t value, which is a signed version of the standard C size_t type. It's size is system dependent, but probably 32-bits on 32-bit systems and 64-bits on 64-bit systems.

The builtin len function uses PyObject_Size, so it has the same limitations. Here's its current implementation:

static PyObject *
builtin_len(PyObject *self, PyObject *v)
{
    Py_ssize_t res;

    res = PyObject_Size(v);
    if (res < 0 && PyErr_Occurred())
        return NULL;
    return PyInt_FromSsize_t(res);
}

You can work around this limitation by using the len method on the IP object, rather than calling the builtin len:

IP('2001::/64').len()

This is pure python, so it doesn't have any limitations on integer size.