How To Decrypt AWS Ruby Client-side Encryption in Python

Question

AWS's S3 SDK for Ruby allows for client-side ('envelope') encryption of the file. It's a combination of AES CBC/ECB encryption for the client-side key where the envelope key and initialization vector are stored in the metadata.

I have a Ruby developer that has encrypted various files in an S3 bucket that I need to retrieve and decrypted with Python. The Python S3 AWS SDK doesn't currently have this client-side feature.

Assuming the file was encrypted using the encryption_key parameter of the Ruby bucket.write S3 API:

#!/usr/bin/ruby
# coding: utf-8

require 'aws-sdk'
require 'openssl'

access_key = '<aws_access_key>'
secret_access_key = '<secret_access_key>'
encryption_key = "passwordpassword"

s3 = AWS::S3.new
storage_host = "our_bucket"
storage_path = "prod/master_report/test.txt"
bucket_obj = s3.buckets[storage_host].objects[storage_path]
bucket_obj.write(file: 'test.txt', encryption_key: encryption_key)

Is there a way to decrypt the files with Python instead of using the Ruby SDK?

Answer 1

Took a bit of tracing through the Ruby S3 SDK but client side-encryption is implemented as "envelope encryption" using the AES algorithm. In short the contents of the envelope are encrypted using AES-CBC with the key and IV stored in the file's metadata (base64 encoded). The CBC key itself is AES-EBC encoded with the users given encryption key.

The Ruby client-side encrypted file can be decrypted with this Python:

#!/usr/bin/python
# -*- coding: utf-8 -*-

from __future__ import print_function
import boto
import tempfile
import base64
from Crypto.Cipher import AES

ENCRYPTION_KEY = b"passwordpassword"
ENV_KEY_LENGTH = 32

conn = boto.connect_s3()
bucket = conn.get_bucket('our_bucket', validate=False)
encrypted = bucket.get_key('prod/master_report/test.txt')

# get envelop keys from file metadata
metadata = encrypted.metadata
envelope_key = base64.b64decode(metadata['x-amz-key'])
envelope_iv = base64.b64decode(metadata['x-amz-iv'])

# decrypt envelope key
cipher = AES.new(ENCRYPTION_KEY, AES.MODE_ECB)
envelope_key = cipher.decrypt(envelope_key)[:ENV_KEY_LENGTH]

# write encrypted file
tf = tempfile.TemporaryFile()
encrypted.get_file(tf)

cipher = AES.new(envelope_key, AES.MODE_CBC, envelope_iv)

# read contents of file
contents = ""
with open('simple/decrypted.txt', 'w') as out:
    tf.seek(0)
    with tf:
        for line in tf:
            dec_line = cipher.decrypt(line)
            contents += dec_line
            print(dec_line, file=out)

tf.close()

print("Decrypted: %s" % (contents,))