How to find un encrypted file in Amazon AWS S3 bucket?

Question

What i have: several old s3 buckets with 1M objects in each, with server-side encryption turned on.

Problem: old files are unencrypted. And i can't say when encryption was turned on. So, i need to find all unencrypted files.

I've tried solution with awscli, but it is pretty slow - 1 request in 2 seconds.

my solution:

s3_buckets="uploads tmp logs whatever "
for s3_bucket in $s3_buckets;
do
    aws s3 ls s3://$s3_bucket --recursive \
    | awk '{print $NF}' \
    | ( while read object ; 
        do 
            object_status=$(aws s3api head-object --bucket $s3_bucket --key $object --query ServerSideEncryption --output text 2>&1) 
            if [ "$object_status" != "AES256" ]; then
                echo "Unencrypted object $object in s3://$s3_bucket"; >> /tmp/body.tmp
                objects_unencrypted=$((objects_unencrypted + 1))
            fi
            objects_count=$((objects_count + 1))
        done
    echo "Bucket $s3_bucket has $objects_count, where unencrypted $objects_unencrypted." >> /tmp/body.tmp )
done

so, maybe there are any better solutions?

is it possible to create Cloudwatch metric to show unencrypted fiels? or any others?

Answer 1

Use Amazon S3 Inventory.

The inventory list contains a list of the objects in an S3 bucket and the metadata for each listed object includes, among other things:

• Encryption status – Set to SSE-S3, SSE-C, SSE-KMS, or NOT-SSE. The server-side encryption status for SSE-S3, SSE-KMS, and SSE with customer-provided keys (SSE-C). A status of NOT-SSE means that the object is not encrypted with server-side encryption.