Trying to set up Amazon's S3 bucket: 403 Forbidden error & setting permissions

Question

I'm following Hartl's railstutorial.org and have arrived at 11.4.4: Image upload in production. What I've done:

• Signed up for Amazon Web Services
• In Amazon Identity and Access Management I created a user. The user was created succesfully.
• In Amazon S3 I created a new bucket.
• Set permissions on the new bucket:

Permissions: The tutorial instructs to "grant read and write permission to the user created in the previous step". However, under 'Permissons' for the bucket the new user's name is not mentioned. I could only choose between Everyone, Authenticated users, Log delivery, Me, and a username Amazon seems to have created itself from my name + a number. I have tried it by selecting Authenticated users and checked the boxes for Upload/Delete and View permissions (and not selecting List and Edit permissions). I also tried it by selecting Everyone and checked all the four boxes. The results were the same.

• I implemented in my Rails application following the instructions of the tutorial (I don't belief anything wen't wrong there, so I haven't included those steps here to not make it too long).

The initializer carrier_wave.rb is set to the code below. I've added region: 'eu-west-1' to the initializer (an idea I got from here) to get rid of the message connecting to the matching region will be more performant.

if Rails.env.production?
  CarrierWave.configure do |config|
    config.fog_credentials = {
      # Configuration for Amazon S3
      :provider              => 'AWS',
      :aws_access_key_id     => ENV['S3_ACCESS_KEY'],           # Set these key's using heroku config:set S3_ACCESS_KEY=<access key>
      :aws_secret_access_key => ENV['S3_SECRET_KEY'],
      :region                => 'eu-west-1'
    }
    config.fog_directory     =  ENV['S3_BUCKET']
  end
end

• It worked in development and I pushed to Heroku.

Error: When using the uploader in production to upload an image, I get the error We're sorry, but something went wrong. The Heroku server log says:

app[web.1]:   SQL (1.7ms)  UPDATE "users" SET "avatar" = $1, "updated_at" = $2 WHERE "users"."id" = $3  [["avatar", "animaatjes.png.gif"], ["updated_at", "2015-05-20 12:37:56.683858"], ["id", 18]]
heroku[router]: at=info method=POST path="/users/18" host=xxx.herokuapp.com request_id=xxx-7f9f-4580-89ba-xxx fwd="xx.xxx.xx.xxx" dyno=web.1 connect=0ms service=3461ms status=500 bytes=1714
app[web.1]: Completed 500 Internal Server Error in 3317ms (ActiveRecord: 13.0ms)
app[web.1]:    (1.4ms)  ROLLBACK
app[web.1]: Excon::Errors::Forbidden (Expected(200) <=> Actual(403 Forbidden)
app[web.1]: excon.error.response
app[web.1]:   :body          => "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>F8xxxD89</RequestId><HostId>MdB5iSMgxxx1vqE+Q=</HostId></Error>"
app[web.1]:   :headers       => {
app[web.1]:     "Connection"       => "close"
app[web.1]:     "Content-Type"     => "application/xml"
app[web.1]:     "Date"             => "Wed, 20 May 2015 12:37:57 GMT"
app[web.1]:     "Server"           => "AmazonS3"
app[web.1]:     "x-amz-id-2"       => "MdB5iSMg***K1vqdP+E+Q="
app[web.1]:     "x-amz-request-id" => "F80A**C58"
app[web.1]:   }
app[web.1]:   :local_address => "***.**.**.**"
app[web.1]:   :local_port    => *****
app[web.1]:   :reason_phrase => "Forbidden"
app[web.1]:   :remote_ip     => "**.***.***.***"
app[web.1]:   :status        => 403
app[web.1]:   :status_line   => "HTTP/1.1 403 Forbidden\r\n"
app[web.1]:   app/controllers/users_controller.rb:46:in 'update'
app[web.1]: Completed 500 Internal Server Error in 6151ms (ActiveRecord: 60.7ms) 

What have I done wrong? I seems to have something to do with the permissions?

---

Update: The cause turned out to be the policies granted to the user. If I grant the user standard AmazonS3FullAccess then it works. It doesn't work with just AmazonS3ReadOnlyAccess, since then the user can't save a new image. In my application the user basically only needs 2 rights: upload its own avatar image and read the avatar image. Would it be safe to use AmazonS3FullAccess or should I write my own custom policy?

I tried the custom policy below, which should give the application read and write rights (adopted from here) but that still generated the 403 Forbidden error.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:ListBucket"],
      "Resource": ["arn:aws:s3:::mybucket"]
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetObject",
        "s3:DeleteObject"
      ],
      "Resource": ["arn:aws:s3:::mybucket/*"]
    }
  ]
}

Answer 1

To enable S3 file Uploads, I had to:

• specify my region (us-west)
• create an IAM user
• add a Bucket Policy specifying that user as a Principal

{
    "Version": "2008-10-17",
    "Statement": [
        {
            "Sid": "AllowFileUpload",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::XXXXXXX:user/instaswan"
            },
            "Action": [
                "s3:GetObject",
                "s3:PutObject",
                "s3:PutObjectAcl"
            ],
            "Resource": [
                "arn:aws:s3:::instaswan-dev",
                "arn:aws:s3:::instaswan-dev/*"
            ]
        }
    ]
}

Be sure to include both the top-level and "/*" Resource, and include any other "Action" attributes you need.

Answer 2

The root of the problem indeed turned out to be the permissions. It was necessary to write a custom policy and set the CORS configuration on the bucket. For any future users that want to implement this chapter of the rails tutorial, see Writing an IAM policy and CORS configuration for Amazon S3 for the necessary code.

Answer 3

In my case s3:PutObjectAcl was the permission that was missing

Answer 4

You may be missing a step which is to set your IAM policy if using an IAM user to configure your bucket.

Go to IAM console, select your user, go to the permissions tab, click the attach policy button and add administrator access.

After that the error should go away and you can upload files with no problem ;)

Answer 5

I had the same problem. Tried many solutions including S3@FullAccess, creating users, writing custom policies. If someone encounters the problem, then check bucket permissions. Go to bucketname > Permissions > Public Access Settings, then change two ACLs settings from True to False.