Migrating from local paperclip storage to S3

Question

We've recently moved to a new webhost that offers limited storage space so we're trying to move all of our user uploads (Avatars, Files, etc.) uploaded via the Paperclip gem to Amazon S3. I have several hundred files all corresponding to different models that I'm now trying to migrate en masse.

I found this document that introduces a nifty paperclip rake task:

rake paperclip:refresh:missing_styles

This command does some of the work for me, however, I noticed it's only setting up the files structure without sending any data - in addition it's not setting up any thumbnails defined using the :styles hash in the has_attached_file call. I.e., I have the following paperclip setup on one of my models:

class User < ActiveRecord::Base
  has_attached_file :avatar,
    :styles => {
      :thumb => "100x100#",
      :small  => "150x150>",
      :medium => "200x200" }
end

Here's some sample output after running the command:

$ rake paperclip:refresh:missing_styles
Regenerating User -> avatar -> [thumb, :small, :medium]
Regenerating Mercury::Image -> image -> [:medium, :thumb]
Regenerating Profile -> image -> [:home_feature, :large, :medium, :thumb]
Regenerating Page -> preview -> [:portfolio]
Regenerating Category -> default_image -> [:home_feature, :large, :medium, :thumb]

Navigating to my S3 Bucket I can see all the directories are correctly setup and for each attachment, but only for original image files and they're all 0 bytes. Am I misunderstanding the usage of this command? I couldn't find any other tool for uploading entire directories of files in bulk to S3, if there's a safe tool out there that already covers this without requiring payment then I'm open ears. I've tried building a ruby script to plug into their SDK and upload these files manually, but their Ruby documentation isn't great.

Answer 1

Looks like I needed to go beyond ruby on this one, s3cmd seemed to be the most appropriate tool for this sort of job. In my case, the sync command did the trick:

s3cmd sync my-app/public/system/ s3://mybucket