Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Encrypted bucket notifications from S3 to SQS

How can I set up S3 bucket notifications to a queue in SQS where KMS are used on both the bucket and the queue?

  • I have a bucket in S3 where the contents are encrypted with an AWS Managed Key (the aws/s3 default key).
  • I have a queue in SQS where SSE (server-side encryption) is enabled, but using a CMK (Customer-Managed Key).

When I go into the S3 web console and try adding a notification event on my bucket that sends to my queue in SQS, I am presented with this error message:

Unable to validate the following destination configurations. The message to the SSE queue could not be encrypted using KMS. (arn:aws:sqs:ca-central-1: ... : ...)

I've already tried configuring my KMS Key Policy to give the S3 service account the permissions it needs.

        {
            "Sid": "Let S3 encrypt messages so that bucket notifications can be encrypted",
            "Effect": "Allow",
            "Principal": {
                "Service": "s3.amazonaws.com"
            },
            "Action": [
                "kms:GenerateDataKey",
                "kms:Encrypt"
            ],
            "Resource": "*"
        },

What do I need to do in order to allow bucket notifications onto an encrypted queue?

like image 772
Nic Avatar asked May 04 '20 23:05

Nic


People also ask

Can S3 send notification to SQS?

Sends notifications from S3 to SQS when an object is created. This SAM template creates an S3 bucket and SQS queue. S3 writes a messages to the SQS queue when a new object is put into the bucket.

Can S3 write to SNS?

You must grant the Amazon S3 principal the necessary permissions to call the relevant API to publish messages to an SNS topic, an SQS queue, or a Lambda function. This is so that Amazon S3 can publish event notification messages to a destination.

Are SQS messages encrypted?

Amazon SQS server-side encryption uses the 256-bit Advanced Encryption Standard (AES-256 GCM algorithm) to encrypt each message body. The integration with AWS Key Management Service (KMS) allows you to centrally manage the keys that protect SQS messages along with keys that protect your other AWS resources.


1 Answers

According to the documentation, the second action should be kms:Decrypt instead of kms:Encrypt.

It's a bit counter-intuitive as to why kms:Decrypt is needed. You might be thinking that since the SQS queue is encrypted, so when the S3 service calls the SendMessage API, it needs the kms:Encrypt permission to ensure that the message can be encrypted by the KMS service, right? Not quite, and here is why:

  1. The message is not encrypted directly by the master key (CMK). Instead, it uses envelope encryption, and the data is encrypted by the data key. Why? Because the maximum size of the data that a symmetric CMK can encrypt using the Encrypt API is 4096 bytes, while the size of a message that the SQS service supports can be far greater than 4KB. Therefore, kms:Encrypt is not needed, and instead, kms:GenerateDataKey is required to generate the data key which is used to encrypt the SQS message.

  2. In the section "Configure KMS permissions for producers" of this AWS doc, it explains why kms:Decrypt is needed.

The call to kms:Decrypt is to verify the integrity of the new data key before using it.

To be abundantly clear, the GenerateDataKey API returns both the plaintext data key and the encrypted copy of the data key. The plaintext data key is used to encrypt the SQS message and the encrypted data key will be stored along with the message in the queue. The only way to verify that the encrypted data key is indeed the ciphertext of the plaintext data key is that the service needs to have the kms:Decrypt permission to decrypt the ciphertext and make sure the output is exactly the same as the plaintext data key returned in the GenerateDataKey API response.

like image 164
jellycsc Avatar answered Oct 22 '22 16:10

jellycsc