Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Parsing CSV data from memory in Python

Tags:

python

csv

Is there a way to parse CSV data in Python when the data is not in a file? I'm storing CSV data in my database and I'd like to parse it. I'm looking for something analogous to Ruby's CSV.parse. I know Python has a CSV class but everything I've seen in the docs seems to deal with files as opposed to in-memory CSV data.

(And it's not an option to parse the data before it goes into the database.)

(And please don't tell me not to store the CSV data in the database. I know what I'm doing as far as the database goes.)

like image 254
Jason Swett Avatar asked Jan 31 '11 20:01

Jason Swett


People also ask

How do I read a 10gb CSV file in Python?

read_csv(chunksize) One way to process large files is to read the entries in chunks of reasonable size, which are read into the memory and are processed before reading the next chunk. We can use the chunk size parameter to specify the size of the chunk, which is the number of lines.

Can Python parse CSV file?

Parsing CSV files in Python is quite easy. Python has an inbuilt CSV library which provides the functionality of both readings and writing the data from and to CSV files. There are a variety of formats available for CSV files in the library which makes data processing user-friendly.


1 Answers

There is no special distinction for files about the python csv module. You can use StringIO to wrap your strings as file-like objects.

like image 116
sinelaw Avatar answered Sep 22 '22 11:09

sinelaw