AWS SDK v2 AllAccessDisabled error for S3 file copy

Question

I'm in the process of switching over to the new aws-sdk in a rails app I wrote and cannot for the life of me find working corresponding methods in the v2 sdk. I'm also running into access denied issues I can't work out.

The way I make use of the v1 sdk is that users directly upload to s3 using an "uploads" namespaced key, and after they create the object they're working on, a callback moves the file to the longterm key and deletes the old one. Here is an example of that:

  def move_file
    old_key = s3_key
    new_key = "#{self.class.table_name}/#{id}/#{Digest::SHA1.hexdigest([Time.now, rand].join)}/#{filename}"
    AWS.config(access_key_id: ENV['AWS_ACCESS_KEY_ID'], secret_access_key: ENV['AWS_SECRET_ACCESS_KEY'], region: 'us-east-1')
    s3 = AWS::S3.new
    bucket_name = ENV['AWS_S3_BUCKET']
    bucket = s3.buckets[bucket_name]
    object = bucket.objects[old_key]

    begin
      object.move_to new_key, :acl => :public_read
      rescue AWS::S3::Errors::NoSuchKey
          errors.add(:base, "Oops! Something went wrong uploading your file. Please try again, and if the problem persists, open a trouble ticket.")
    end

    if !bucket.objects[old_key].exists? && bucket.objects[new_key].exists?
      update_column(:s3_key, new_key)
    end
  end

Works great, but now I'm trying to update to the new sdk. What I've been trying is this:

  def move_file
    old_key = file
    new_key = "#{self.class.table_name}/#{id}/#{Digest::SHA1.hexdigest([Time.now, rand].join)}/#{filename}"
    s3 = Aws::S3::Client.new

    begin
      s3.copy_object({copy_source:old_key, key:new_key, bucket: ENV['AWS_S3_BUCKET'], acl:'public-read'})
      s3.delete_object({bucket: ENV['AWS_S3_BUCKET'], key:old_key})
      update_column(:file, new_key)
      rescue Aws::S3::Errors::ServiceError
          errors.add(:base, "Oops! Something went wrong uploading your file. Please try again, and if the problem persists, open a trouble ticket.")
    end
  end

Whenever I try to move the uploaded file it throws and error - Aws::S3::Errors::AllAccessDisabled: All access to this object has been disabled

I have tried changing the way I handle security credentials. Instead of a naked access key/ secret key pair, I created a user in IAM, attached a policy that grants them full access to S3 and tried using those credentials, to no avail.

What am I doing wrong? But also, if anyone is familiar with the new sdk, is my copy_object approach even correct?

Answer 1

The error is caused by the :copy_source value you are passing to #copy_object. This value must be the source bucket and source key, separated by a slash (/):

"#{sourcebucket}/#{sourcekey}"

Your old_key value contains a forward slash. Amazon S3 is taking the first path segment of that key and treating it as a bucket name. Because you do not have permission to that bucket, you are getting an auth error. Your credential configuration is probably just fine.

To correct this error:

def move_file
  bucket = ENV["AWS_S3_BUCKET"]
  old_key = file
  new_key = "#{self.class.table_name}/#{id}/#{Digest::SHA1.hexdigest([Time.now, rand].join)}/#{filename}"
  s3 = Aws::S3::Client.new

  begin
    s3.copy_object(bucket:bucket, key:new_key, copy_source:"#{bucket}/#{old_key}", acl:'public-read')
    s3.delete_object(bucket:bucket, key:old_key)
    update_column(:file, new_key)
  rescue Aws::S3::Errors::ServiceError
    errors.add(:base, "Oops! Something went wrong uploading your file. Please try again, and if the problem persists, open a trouble ticket.")
  end
end