Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Different S3 buckets for staging app vs. production app

I have a Rails app that uses Amazon S3 to store Paperclip attachments. My database.yml file specifies different S3 buckets for development, test, and production.

I have two apps on Heroku - a production app and a 'staging app' for testing code on the platform pre-launch.

The current system has an important flaw - it uses the same S3 bucket for staging and production. How can I set up my system so it uses different buckets depending essentially on whether I write git push production master or git push staging master?

like image 975
sscirrus Avatar asked Aug 25 '11 21:08

sscirrus


People also ask

Do S3 buckets need to be unique?

Amazon S3 supports global buckets, which means that each bucket name must be unique across all AWS accounts in all the AWS Regions within a partition.

What are the types of S3 buckets in AWS?

S3 Storage Classes can be configured at the object level, and a single bucket can contain objects stored across S3 Standard, S3 Intelligent-Tiering, S3 Standard-IA, and S3 One Zone-IA.

Can a S3 bucket store multiple versions of the same file?

You can use S3 Versioning to keep multiple versions of an object in one bucket so that you can restore objects that are accidentally deleted or overwritten.

Can we have two S3 buckets with the same name?

(i.e. No two S3 buckets can have the same name.) It's similar to how DNS works where each domain name must be unique. Therefore, you need to use a unique bucket name when creating S3 buckets.


2 Answers

Heroku permits you to configure anything you like via persistent environment variables that every dyno/process in an application gets started with. Environment variables are not shared between staging and production versions of the same application. Take advantage of this.

has_attached_file :photo, 
  :styles => ...,
  :path => ...,
  :storage => :s3,
  :bucket => ENV['S3_BUCKET'], # <--- over here
  :s3_credentials => {
    :access_key_id => ENV['S3_KEY'],
    :secret_access_key => ENV['S3_SECRET']
  }

Then:

# Configure the "staging" instance
$ heroku config:add \
    RACK_ENV=production \
    S3_KEY=my-staging-key \
    S3_SECRET=my-staging-secret \
    S3_BUCKET=my-staging-bucket \
    --app my-staging-app-name

# Configure the "production" instance
$ heroku config:add \
    RACK_ENV=production \
    S3_KEY=my-production-key \
    S3_SECRET=my-production-secret \
    S3_BUCKET=my-production-bucket \
    --app my-production-app-name

Note that each instance of your application has a RACK_ENV=production. Don't use Rails environments for differentiating between instances of your application. Rather, your application should expect that environment variables be used to configure instance-specific aspects.

like image 134
yfeldblum Avatar answered Sep 21 '22 09:09

yfeldblum


Set config:add RACK_ENV=staging on your staging box, and then in your code you can specify the bucket depending on the environment. Eg:

if Rails.env.production?
    has_attached_file :photo, 
    :styles => ...,
    :path => ...,
    :storage => :s3,
    :bucket => 'your_prod_bucket',
    :s3_credentials => {
      :access_key_id => ENV['S3_KEY'],
      :secret_access_key => ENV['S3_SECRET']
    }
  else
    has_attached_file :photo, 
    :styles => ...,
    :path => ...,
    :storage => :s3,
    :bucket => 'your_staging_bucket',
    :s3_credentials => {
      :access_key_id => ENV['S3_KEY'],
      :secret_access_key => ENV['S3_SECRET']
    }

  end

This heroku post also seem to suggest that you can achieve this simply by using different s3 credentials for staging and production. I'm guessing you'd have to do some configuration on amazon's side as well. Anyways, take a look to see if it helps. http://devcenter.heroku.com/articles/config-vars

like image 45
Kevin Tsoi Avatar answered Sep 21 '22 09:09

Kevin Tsoi