python bit shifting with numpy

Question

I'm working with 64 bit unsigned integers and after bit shifting comparing the value before decoding the rest of the bit values. I'm iterating over millions of values and trying to minimize process time.

The issue is bit shifting is not supported with uint64 nor numpy-uint64. I'm trying to avoid using int64 to avoid negative values.

example data: 0x8204000000000080 after shifting(word>> 60): =-8 #but comparing to 0x8

looping one million times and seeing how long it takes it was found that of all methods the '>>' shift operator was the most expedient with the next best option to call the abs() function. Is there a better more expedient solution for this?

Loop code:

import numpy as np
import time

start_time= time.time()
for i in range(1000000):
    x= np.int64(-1)
    x=np.right_shift(x,60)
print (time.time()-start_time)

start_time= time.time()
for i in range(1000000):
    x= np.uint64(-1)
    x=int(x/(2**60))
print (time.time()-start_time)

start_time= time.time()
for i in range(1000000):
    x= np.int64(-1)
    x=abs(x>>60)
print (time.time()-start_time)

start_time= time.time()
for i in range(1000000):
    x= np.int64(-1)
    x= x>>60
print (time.time()-start_time)

Output:

2.055999994277954
3.1540000438690186
0.619999885559082
0.5810000896453857

Answer 1

The issue is that when you apply the shift to an array scalar, NumPy tries to produce an output type that can hold all values of both input dtypes (with a Python int cast to either int32 or int64). There is no integer dtype that can hold all values of both uint64 and a signed dtype, and floats aren't an option here.

When one operand is an array and the other is a scalar (here a Python int), NumPy attempts to stuff the scalar into a smaller dtype, which for most shift operations means the shift amount is cast to int8 or uint8, depending on whether the other operand is signed. uint64 and uint8 both fit in uint64.

You'll have to cast the shift amount to an unsigned int:

>>> numpy.uint64(-1) >> 1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: ufunc 'right_shift' not supported for the input types, and the inputs
 could not be safely coerced to any supported types according to the casting rul
e ''safe''
>>> numpy.uint64(-1) >> numpy.uint64(1)
9223372036854775807

Answer 2

>>> import numpy
>>> a = numpy.array([1,2,3],dtype=numpy.uint64)
>>> a>>2
array([0, 0, 0], dtype=uint64)

>>> a = numpy.array([1,2,2**64-1],dtype=numpy.uint64)
>>> a>>2 
array([0, 0, 4611686018427387903], dtype=uint64)
>>> a>>60
array([ 0,  0, 15], dtype=uint64)

I dont understand the problem perhaps?