How to read a csv django http response

Question

In a view, I create a Django HttpResponse object composed entirely of a csv using a simply csv writer:

response = HttpResponse(content_type='text/csv')
response['Content-Disposition'] = 'attachment; filename="foobar.csv"'

writer = csv.writer(response)

    table_headers = ['Foo', 'Bar']
    writer.writerow(table_headers)

    bunch_of_rows = [['foo', 'bar'], ['foo2', 'bar2']]
    for row in bunch_of_rows:
        writer.writerow(row)

return response

In a unit test, I want to test some aspects of this csv, so I need to read it. I'm trying to do so like so:

response = views.myview(args)

reader = csv.reader(response.content)

headers = next(reader)
row_count = 1 + sum(1 for row in reader)

self.assertEqual(row_count, 3) # header + 1 row for each attempt
self.assertIn('Foo', headers)

But the test fails with the following on the headers = next(reader) line:

nose.proxy.Error: iterator should return strings, not int (did you open the file in text mode?)

I see in the HttpResponse source that response.content is spitting the string back out as a byte-string, but I'm not sure the correct way to deal with that to let csv.reader read the file correctly. I thought I would be able to just replace response.content with response (since you write to the object itself, not it's content), but that just resulted in a slight variation in the error:

_csv.Error: iterator should return strings, not bytes (did you open the file in text mode?)

Which seems closer but obviously still wrong. Reading the csv docs, I assume I am failing to open the file correctly. How do I "open" this file-like object so that csv.reader can parse it?

Answer 1

response.content provides bytes. You need to decode this to a string:

foo = response.content.decode('utf-8')

Then pass this string to the csv reader using io.StringIO:

import io
reader = csv.reader(io.StringIO(foo))

Answer 2

You can use io.TextIOWrapper to convert the provided bytestring to a text stream:

import io
reader = csv.reader(io.TextIOWrapper(io.BytesIO(response.content), encoding='utf-8'))

This will convert the bytes to strings as they're being read by the reader.