Upload and parse csv file with google app engine

Question

I'm wondering if anyone with a better understanding of python and gae can help me with this. I am uploading a csv file from a form to the gae datastore.

class CSVImport(webapp.RequestHandler):
  def post(self):
     csv_file = self.request.get('csv_import')
     fileReader = csv.reader(csv_file)
     for row in fileReader:       
       self.response.out.write(row) 

I'm running into the same problem that someone else mentions here - http://groups.google.com/group/google-appengine/browse_thread/thread/bb2d0b1a80ca7ac2/861c8241308b9717

That is, the csv.reader is iterating over each character and not the line. A google engineer left this explanation:

The call self.request.get('csv') returns a String. When you iterate over a string, you iterate over the characters, not the lines. You can see the difference here:

 class ProcessUpload(webapp.RequestHandler): 
   def post(self): 
     self.response.out.write(self.request.get('csv')) 
     file = open(os.path.join(os.path.dirname(__file__), 'sample.csv')) 
     self.response.out.write(file) 

     # Iterating over a file 
     fileReader = csv.reader(file) 
     for row in fileReader: 
       self.response.out.write(row) 

     # Iterating over a string 
     fileReader = csv.reader(self.request.get('csv')) 
     for row in fileReader: 
       self.response.out.write(row) 

I really don't follow the explanation, and was unsuccessful implementing it. Can anyone provide a clearer explanation of this and a proposed fix?

Thanks, August

Answer 1

Short answer, try this:

fileReader = csv.reader(csv_file.split("\n"))

Long answer, consider the following:

for thing in stuff:
  print thing.strip().split(",")

If stuff is a file pointer, each thing is a line. If stuff is a list, each thing is an item. If stuff is a string, each thing is a character.

Iterating over the object returned by csv.reader is going to give you behavior similar to iterating over the object passed in, only with each item CSV-parsed. If you iterate over a string, you'll get a CSV-parsed version of each character.