Working with bit streams

Question

I have a base64 encoded bit stream, I want to work with. After decoding it with base64.b64decode I get a bytes object (Py3k btw) containing the decoded code. The problem is now, that I would like to work on that bytes object with bit-wise operations, such as shifting, bit wise and etc, but that is not possible as it is a bytes sequence.

Is there any way to use binary operations on that decoded code?

Answer 1

try using

list(bytestring)

Eg.

>>> bstring=b"Hello World"
>>> list( bstring)
[72, 101, 108, 108, 111, 32, 87, 111, 114, 108, 100]
>>> 

If you want one huge bitfield instead of all those octets

>>> from functools import reduce
>>> reduce(lambda x,y:(x<<8)+y,list(b"Hello World"))
87521618088882533792115812
>>> bin(_)
'0b100100001100101011011000110110001101111001000000101011101101111011100100110110001100100'
>>> 

You didn't say how you are counting the bits, perhaps they are supposed to be reversed

>>> reduce(lambda x,y:(x<<8)+y,list(b"Hello World"[::-1]))
121404708493354166158910792
>>> bits=bin(_)[2:]

and pad the string to even bytes

>>> bits=bits.zfill(((len(bits)-1)//8+1)*8)
>>> bits
'0110010001101100011100100110111101010111001000000110111101101100011011000110010101001000'

turn the first 6 bits into an int

>>> int(bits[:6],2)
25

and then the following 4 bits

>>> int(bits[6:10],2)
1

Answer 2

If you need to make your bytes object mutable then you can construct a bytearray from it:

mutable = bytearray(b"immutable")

This will let you modify the individual bytes through normal assignments

mutable[0] = mutable[1] = 32

If you need to do bit-wise operations then I suggest trying bitstring (with apologies for recommending my own module). It works for Python 3 and lets you do bit-wise slicing, shifting, logical operations and much more.

>>> s = bitstring.BitArray(bytes=b'your_bytes_object')
>>> s.hex
'0x796f75725f62797465735f6f626a656374'
>>> ten_bits = s[5:15]
>>> print(ten_bits, ten_bits.int)
0b0010110111 183
>>> print(ten_bits << 2)
0b1011011100
>>> print(s[0:6] & '0b110100')
0b010100