How do I parse a CSV file located in a Amazon S3 bucket

Question

Below is the code I'm using to parse the CSV from within the app, but I want to parse a file located in a Amazon S3 bucket. It needs to work when pushed to Heroku as well.

namespace :csvimport do
  desc "Import CSV Data to Inventory."
  task :wiwt => :environment do
    require 'csv'

    csv_file_path = Rails.root.join('public', 'wiwt.csv.txt')

    CSV.foreach(csv_file_path) do |row|
      p = Wiwt.create!({
        :user_id => row[0],
        :date_worn => row[1],
        :inventory_id => row[2],
      })
    end
  end
end

Answer 1

There are cases with S3, when permissions on S3 Object disallow public access. In-built Ruby functions do assume a path is publicly accessible and don't account for AWS S3 specificity.

s3 = Aws::S3::Resource.new
bucket = s3.bucket("bucket_name_here")
str = bucket.object("file_path_here").get.body.string
content = CSV.parse(str, col_sep: "\t", headers: true).map(&:to_h)

Per-line explanation using AWS SDK: Line 1. Initialize Line 2. Choose a bucket. Line 3. Choose an object and get it as a String. Line 4. Effectively CSV.parse('the string'), but I also added a options and map over it just in case it helps you.