Rails Exporting to CSV without constructing CSV in memory

Question

Rails 2.3.5, Ruby 1.8.7.

In a prior question, there are a number of solutions to exporting data in CSV format, but some of them seem to construct the data in memory before sending. Is this wise when exporting large data sets? Do any of the solutions in the prior question avoid this.

Or is it impossible to avoid building a response without building the whole response locally, either in memory or in a temp file?

I would not be surprised if the latter was true, since if there is an error in the CSV generation, you might want to send an error message back instead, but I might be generating way too much data to want to generate the data in memory/on disk.

Answer 1

You can stream directly from the CSV library (FasterCSV in Ruby 1.8).

render :text => proc { |response, output|
    CSV.generate(output) do |csv|
      csv << ...
    end
}

You should also use find_in_batches if you are concerned about your memory footprint.