Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Remove 'b' flag in CSV output

Tags:

python

csv

I am trying to write the outputs to a CSV file in Python 3.4 but the CSV file always contains 'b' flags. For example, b'The text output1', b'The text output2',... I am wondering if there is a way to get rid of the 'b' flags. I understand that this is not an issue in Python 2.X.

Here are the codes that I used

with open('test.csv', 'w') as f:
    writer = csv.DictWriter(f, ['field'], extrasaction='ignore')
    writer.writeheader()
    test_text = mongo.test.find({'text': text})
    for t in test_text
        writer.writerow({i:v.encode('utf') for i,v in t.items()})

Thanks very much

------Updates-----------

Thanks very much for Tim Pietzcker, John Zwinck, and Warren Weckesser providing helpful comments and answers. Per Warren's suggestions, if I change my codes to

import csv

data = [chr(0x03d5) + 'oo', 'b' + chr(0x0101) + 'r']

with open('test.csv', 'w') as f:
    writer = csv.writer(f)
    for item in data:
        writer.writerow([item])

I will get error message

UnicodeEncodeError: 'charmap' codec can't encode character '\u03d5' in position 0: character maps to <undefined>

if I change my codes to

import csv

data = [chr(0x03d5) + 'oo', 'b' + chr(0x0101) + 'r']

with open('test.csv', 'w') as f:
    writer = csv.writer(f)
    for item in data:
        writer.writerow([item.encode('utf')])

I will get outputs with 'b' flags

b'\xcf\x95oo'
b'b\xc4\x81r'

Any thoughts on how this is happening and how I might be able to fix it? Thanks again.

------Updates 2-----------

Thanks very much for Warren's solution. The following codes worked!

import csv

data = [chr(0x03d5) + 'oo', 'b' + chr(0x0101) + 'r']

with open('test.csv', 'w', encoding='utf8') as f:
    writer = csv.writer(f)
    for item in data:
        writer.writerow([item])
like image 725
frostman Avatar asked Oct 07 '14 05:10

frostman


People also ask

How do I remove blank lines from a CSV file?

Import Pandas. Read CSV File. Use pop() function for removing or deleting rows or columns from the CSV files. Print Data.

What is Writerow in csv?

writer() function is used to create a writer object. The writer. writerow() function is then used to write single rows to the CSV file.

Why csv writer adds blank rows?

Why csv writer adds blank rows? The way Python handles newlines on Windows can result in blank lines appearing between rows when using csv. writer . In Python 2, opening the file in binary mode disables universal newlines and the data is written properly.


1 Answers

Don't explicitly encode the strings yourself; let the writer take care of it. For example, this code:

import csv

data = [chr(0x03d5) + 'oo', 'b' + chr(0x0101) + 'r']

with open('test.csv', 'w') as f:
    writer = csv.writer(f)
    for item in data:
        writer.writerow([item])

writes the file

ϕoo
bār

with UTF-8 encoding (at least it does on my system, where locale.getpreferredencoding(False) returns 'UTF-8'). To make the encoding explicit, you can set the encoding in the call to open:

    with open('test.csv', 'w', encoding='utf8') as f:

If the last line is changed to writer.writerow([item.encode('utf')]) (which converts the strings to bytes), it produces

b'\xcf\x95oo'
b'b\xc4\x81r'

In your example, try changing this line:

        writer.writerow({i:v.encode('utf') for i,v in t.items()})

to this:

        writer.writerow(t)

Then if that works, you could replace this:

    for t in test_text
        writer.writerow({i:v.encode('utf') for i,v in t.items()})

with

    writer.writerows(test_text)
like image 125
Warren Weckesser Avatar answered Oct 20 '22 06:10

Warren Weckesser