Decompress bz2 files

Question

I would like to decompress the files in different directories which are in different routes. And codes as below and the error is invalid data stream. Please help me out. Thank you so much.

import sys import os import bz2 from bz2 import decompress  path = "Dir" for(dirpath,dirnames,files)in os.walk(path):    for file in files:        filepath = os.path.join(dirpath,filename)        newfile = bz2.decompress(file)        newfilepath = os.path.join(dirpath,newfile) 

Answer 1

bz2.compress/decompress work with binary data:

>>> import bz2 >>> compressed = bz2.compress(b'test_string') >>> compressed b'BZh91AY&SYJ|i\x05\x00\x00\x04\x83\x80\x00\x00\x82\xa1\x1c\x00 \x00"\x03h\x840" P\xdf\x04\x99\xe2\xeeH\xa7\n\x12\tO\x8d \xa0' >>> bz2.decompress(compressed) b'test_string' 

In short - you need to process file contents manually. In case you have very large files you should prefer using bz2.BZ2Decompressor to bz2.decompress, because the latter requires that you store the entire file in a byte array.

for filename in files:     filepath = os.path.join(dirpath, filename)     newfilepath = os.path.join(dirpath,filename + '.decompressed')     with open(newfilepath, 'wb') as new_file, open(filepath, 'rb') as file:         decompressor = BZ2Decompressor()         for data in iter(lambda : file.read(100 * 1024), b''):             new_file.write(decompressor.decompress(data)) 

You can also use bz2.BZ2File to make this even simpler:

for filename in files:     filepath = os.path.join(dirpath, filename)     newfilepath = os.path.join(dirpath, filename + '.decompressed')     with open(newfilepath, 'wb') as new_file, bz2.BZ2File(filepath, 'rb') as file:         for data in iter(lambda : file.read(100 * 1024), b''):             new_file.write(data)