Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Can csv data be made lazy?

Using Python's csv module, is it possible to read an entire, large, csv file into a lazy list of lists?

I am asking this, because in Clojure there are csv parsing modules that will parse a large file and return a lazy sequence (a sequence of sequences). I'm just wondering if that's possible in Python.

like image 395
octopusgrabbus Avatar asked Jun 19 '12 21:06

octopusgrabbus


2 Answers

Unless I'm misunderstanding you, this is the default behavior, which is the very essence of reading through a csv file:

import csv

def lazy(csvfile):
    with open(csvfile) as f:
        r = csv.reader(f)
        for row in r:
            yield row

gives you back one row at a time.

like image 63
Tim Pietzcker Avatar answered Sep 28 '22 08:09

Tim Pietzcker


The csv module's reader is lazy by default.

It will read a line in at a time from the file, parse it to a list, and return that list.

like image 25
Gareth Latty Avatar answered Sep 28 '22 07:09

Gareth Latty