Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Slice indices limited to 0x7FFFFFFF

I'm playing with slices in Python (2.7.4):

class Foo():
    def __getitem__(self, key):
        # Single value
        if isinstance(key, (int,long)):
            return key

        # Slice
        if isinstance(key, slice):
            print 'key.start = 0x{0:X}   key.stop = 0x{1:X}'.format(key.start, key.stop)
            length = key.stop - key.start
            return str(length)

Everything seems to work as expected:

>>> f = Foo()
>>>
>>> f[42]
42
>>>
>>> f[20:30]
key.start = 0x14   key.stop = 0x1E
'10'

Except it seems that the slice indices are limited to 0x7FFFFFFF:

>>> f[0xFEDCBA98 : 0xFFFFFFFF]
key.start = 0x7FFFFFFF   key.stop = 0x7FFFFFFF
'0'
>>> f[0x80000000:0x90000000]
key.start = 0x7FFFFFFF   key.stop = 0x7FFFFFFF
'0'

Why are slice indices not subject to the same long integer promotion as regular int values? Is there any workaround for this?

like image 597
Jonathon Reinhart Avatar asked Oct 31 '13 02:10

Jonathon Reinhart


2 Answers

I've realized that this appears to be a limitation of old-style classes. New-style classes (ones that derive from object) behave as expected:

class Foo(object):
   #...

Results:

>>> f = Foo()
>>>
>>> f[0x80000000:0x90000000]
key.start = 0x80000000   key.stop = 0x90000000
'268435456'
>>>
>>> f[0xFEDCBA98 : 0x1200000000]
key.start = 0xFEDCBA98   key.stop = 0x1200000000
'73033532776'
>>>

I haven't seen this documented anywhere. It's especially confusing, since this limitation is not in the slice class itself:

>>> s = slice(0xFEDCBA98, 0x1200000000)
>>>
>>> s
slice(4275878552L, 77309411328L, None)
>>>
>>> hex(s.start)
'0xfedcba98L'
like image 136
Jonathon Reinhart Avatar answered Nov 01 '22 09:11

Jonathon Reinhart


After a lot of searching I found this

In python 3.3, the slice start and end are found like this

start = PyLong_FromSsize_t(istart);
...
end = PyLong_FromSsize_t(istop);

but in 2.7 they are found like this

start = PyInt_FromSsize_t(istart);
...
end = PyInt_FromSsize_t(istop);

In 2.7, PyInt_FromSsize_t ultimately uses the size of long where as in 3.3, PyLong_FromSsize_t uses the size of PyObject. That is why it works fine in 3.3 and not in 2.7.

like image 35
thefourtheye Avatar answered Nov 01 '22 08:11

thefourtheye