what exactly is file format that looks like b'\x1f\x8b\.....'

Question

When I get a file from aws s3 I get it in format as follows:

b'\x1f\x8b\x08\x00\x0e@\xfd[\x00\x03\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00'

What exactly is this?

Also when getting data -unzip - upload back to s3 it can be done by using:

s3 = boto3.client('s3', use_ssl=False)  
s3.upload_fileobj(
    Fileobj = gzip.GzipFile(
                None,
                'rb',
                fileobj=BytesIO(s3.get_object(Bucket=bucket, Key=gzipped_key)['Body'].read())),
    Bucket=bucket,
    Key=uncompressed_key)

I get b'\x1f\x8b\x08\x00\x0e@\xfd[\x00\x03\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00' when I run s3.get_object(Bucket=bucket) line but why wrap that around BytesIO?

Answer 1

What exactly is this?

[0x1f, 0x8b, 0x08] is the header for a gzip file. I suggest you run it through gzip or open it with something like 7zip to examine its contents.

What about BytesIO?

Good question. Unfortunately there's nowhere near enough information in that question to give you a concise answer. If you wanted a 5000-word essay on BytesIO and how it all works, I'd be happy to oblige, but this probably isn't the right place for it :-)

---

On a more serious note, you may want to narrow your area of interest down to something a little more specific. In addition, it's usually a good idea to limit questions to one question, since that makes StackOverflow much easier to search/manage.