Trying to generate a presigned url link so an user can download an Amazon S3 object, but getting invalid request

Question

I am currently using the Ruby aws-sdk, version 2 gem with server-side customer-provided encryption key(SSE-C). I am able to upload the object from a rails form to Amazon S3 with no issues.

def s3
  Aws::S3::Object.new(
    bucket_name: ENV['S3_BUCKET'],
    key: 'hello',
  )
end

def upload_object
  customer_key = OpenSSL::Cipher::AES.new(256, :CBC).random_key
  customer_key_md5 = Digest::MD5.new.digest(customer_key)
  object_key = 'hello'
  options = {}
  options[:key] = object_key
  options[:sse_customer_algorithm] = 'AES256'
  options[:sse_customer_key] = customer_key
  options[:sse_customer_key_md5] = customer_key_md5
  options[:body] = 'hello world'
  options[:bucket] = ENV['S3_BUCKET']
  s3.put(options)
  test_params = {
    object_key: object_key,
    customer_key: Base64.encode64(customer_key),
    md5_key: Base64.encode64(customer_key_md5),
  }
  Test.create(test_params)
end

But I'm having some issues with retrieving the object and generating a signed url link for the user to download it.

def retrieve_object(customer_key, md5)
  options = {}
  options[:key] = 'hello
  options[:sse_customer_algorithm] = 'AES256'
  options[:sse_customer_key] = Base64.decode64(customer_key)
  options[:sse_customer_key_md5] = Base64.decode64(md5)
  options[:bucket] = ENV['S3_BUCKET']
  s3.get(options)
  url = s3.presigned_url(:get)
end

The link is generated, but when I clicked on it, it directs me to an amazon page saying.

<Error>
<Code>InvalidRequest</Code>
<Message>
The object was stored using a form of Server Side Encryption. The correct   parameters must be provided to retrieve the object.
</Message>
<RequestId>93684EEBA062B1C2</RequestId>
<HostId>
OCnn5EG7ydfoKzsmEDMbqK5kOhLFpNXxVRdekfhOfnBc6s+jtPYFsKi8IZsEPcd9ConbYUHgwC8=
</HostId>
</Error>

The error message is not helping as I'm unsure what parameters I need to add. I think I might be missing some permissions parameters.

Get Method http://docs.aws.amazon.com/sdkforruby/api/Aws/S3/Object.html#get-instance_method

Presigned_Url Method http://docs.aws.amazon.com/sdkforruby/api/Aws/S3/Object.html#presigned_url-instance_method

Answer 1

When you generate a pre-signed GET object URL, you need to provide all of the same params that you would pass to Aws::S3::Object#get.

s3.get(sse_customer_algorithm: 'AES256', sse_customer_key: customer_key).body.read

This means you need to pass the same sse_customer_* options to #presigned_url:

url = obj.presigned_url(:get,
  sse_customer_algorithm: 'AES256',
  sse_customer_key: customer_key)

This will ensure that the SDK correctly signs the headers that Amazon S3 expects when you make the final GET request. The next problem is that you are now responsible for sending those values along with the GET request as headers. Amazon S3 will not accept the algorithm and key in the query string.

uri = URI.parse(url)
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
request = Net::HTTP::Get.new(uri.request_uri, {
  "x-amz-server-side-encryption-customer-algorithm" => 'AES256',
  "x-amz-server-side-encryption-customer-key" => Base64.encode64(cpk),
  "x-amz-server-side-encryption-customer-key-MD5" => Base64.encode64(OpenSSL::Digest::MD5.digest(cpk))
})

Please note - while testing this, I found a bug in the presigned URL implementation of the current v2.0.33 version of the aws-sdk gem. This has been fixed now and should be part of v2.0.34 once it releases.

See the following gitst for a full example that patches the bug and demonstrates:

• Uploads an object using a cpk
• Gets the object using the SDK
• Generates a presigned GET url
• Downloads the object using just Net::HTTP and the presigned URL

You can view the sample script here:

https://gist.github.com/trevorrowe/49bfb9d59f83ad450a9e

Just replace the bucket_name and object_key variables at the top of the script.