Using csvreader against a gzipped file in Python

Question

I have a bunch of gzipped CSV files that I'd like to open for inspection using Python's built in CSV reader. I'd like to do this without having first to manually unzip them to disk. I guess I want to somehow get a stream to the uncompressed data, and pass this into the CSV reader. Is this possible in Python?

Answer 1

Use the gzip module:

with gzip.open(filename, mode='rt') as f:     reader = csv.reader(f)     #... 

Answer 2

I've tried the above version for writing and reading and it didn't work in Python 3.3 due to "bytes" error. However, after some trial and error I could get the following to work. Maybe it also helps others:

import csv import gzip import io   with gzip.open("test.gz", "w") as file:     writer = csv.writer(io.TextIOWrapper(file, newline="", write_through=True))     writer.writerow([1, 2, 3])     writer.writerow([4, 5, 6])  with gzip.open("test.gz", "r") as file:     reader = csv.reader(io.TextIOWrapper(file, newline=""))     print(list(reader)) 

As amohr suggests, the following works as well:

import gzip, csv  with gzip.open("test.gz", "wt", newline="") as file:     writer = csv.writer(file)     writer.writerow([1, 2, 3])     writer.writerow([4, 5, 6])  with gzip.open("test.gz", "rt", newline="") as file:     reader = csv.reader(file)     print(list(reader))