Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to write in .csv file from a generator in python

My Generator looks like the following:

def mygen(reader):
    for row in reader:
        yield row[0],row[1],row[2],row[3],row[4]

I'm trying to insert these generator produced values in the following manner:

file1=open(f2,"w")
writes=csv.writer(file1,delimiter=' ',quoting=csv.QUOTE_ALL)
g=mygen(reader)
for x in g:
    writes.writerow([x])

It enters blank rows to the file without producing any output. It works perfectly when I use the same generator to insert code to a table.

like image 956
Parisa Rai Avatar asked Dec 16 '15 10:12

Parisa Rai


People also ask

How do I write data into a CSV file in Python?

Write CSV files with csv.DictWriter() class can be used to write to a CSV file from a Python dictionary. Here, file - CSV file where we want to write to. fieldnames - a list object which should contain the column headers specifying the order in which data should be written in the CSV file.

How do you write data to CSV file in Python using pandas?

By using pandas. DataFrame. to_csv() method you can write/save/export a pandas DataFrame to CSV File. By default to_csv() method export DataFrame to a CSV file with comma delimiter and row index as the first column.

How do you read and write CSV files in Python?

At first, the CSV file is opened using the open() method in 'r' mode(specifies read mode while opening a file) which returns the file object then it is read by using the reader() method of CSV module that returns the reader object that iterates throughout the lines in the specified CSV document.


1 Answers

Your x is already a sequence, you don't need to wrap it in a list:

for x in g:
    writes.writerow(x)

You could just pass your generator directly to writes.writerows():

with open(f2, "w") as file1:
    writes = csv.writer(file1, delimiter=' ', quoting=csv.QUOTE_ALL)
    writes.writerows(mygen(reader))

Demo:

>>> import sys
>>> import csv
>>> def mygen(reader):
...     for row in reader:
...         yield row[0],row[1],row[2],row[3],row[4]
...
>>> reader = csv.reader('''\
... foo,bar,baz,spam,ham,eggs
... 42,81,13,97,73,100
... '''.splitlines())
>>> writes = csv.writer(sys.stdout, delimiter=' ', quoting=csv.QUOTE_ALL)
>>> writes.writerows(mygen(reader))
"foo" "bar" "baz" "spam" "ham"
"42" "81" "13" "97" "73"

You do need to watch out that your reader is not a generator you already exhausted. If it is a csv.reader() object, and you read all rows before, you'll need to re-wind the underlying file object back to the start with fileobj.seek(0). It'd be better to avoid reading the file twice, however.

If your reader object produces sequences (such as lists or tuples), you could just use slicing to get the first 5 elements, no need to type out every single index:

def mygen(reader):
    for row in reader:
        yield row[:5]
like image 182
Martijn Pieters Avatar answered Oct 23 '22 08:10

Martijn Pieters