Reading and interpreting data from a binary file in Python

Question

I want to read a file byte by byte and check if the last bit of each byte is set:

#!/usr/bin/python  def main():     fh = open('/tmp/test.txt', 'rb')     try:         byte = fh.read(1)         while byte != "":             if (int(byte,16) & 0x01) is 0x01:                 print 1             else:                 print 0             byte = fh.read(1)     finally:         fh.close      fh.close()  if __name__ == "__main__":         main() 

The error I get is:

Traceback (most recent call last):   File "./mini_01.py", line 21, in <module>     main()   File "./mini_01.py", line 10, in main     if (int(byte,16) & 0x01) is 0x01: ValueError: invalid literal for int() with base 16: '\xaf' 

Anyone an idea? I didn't succeed using the struct and the binascii modules.

Answer 1

Try using the bytearray type (Python 2.6 and later), it's much better suited to dealing with byte data. Your try block would be just:

ba = bytearray(fh.read()) for byte in ba:     print byte & 1 

or to create a list of results:

low_bit_list = [byte & 1 for byte in bytearray(fh.read())] 

This works because when you index a bytearray you just get back an integer (0-255), whereas if you just read a byte from the file you get back a single character string and so need to use ord to convert it to an integer.

---

If your file is too big to comfortably hold in memory (though I'm guessing it isn't) then an mmap could be used to create the bytearray from a buffer:

import mmap m = mmap.mmap(fh.fileno(), 0, access=mmap.ACCESS_READ) ba = bytearray(m) 

Answer 2

You want to use ord instead of int:

if (ord(byte) & 0x01) == 0x01: