How to cope with the performance of generating signed URLs for accessing private content via CloudFront?

Question

A common use case of AWS S3 and CloudFront is serving private content. The common solution is using signed CloudFront URLs to access private files stored using S3.

However, the generation of these URLs comes with a cost: computing the RSA signature of any given URL using a private key. For Python (or boto, AWS's Python SDK), the rsa (https://pypi.python.org/pypi/rsa) library is used for this task. On my late 2014 MBP, it takes about ~25ms per computation with a 2048-bit key.

This cost potentially impacts the scalability of an application that uses this approach for authorizing access to private content via CloudFront. Imagine multiple clients request for access to multiple files frequently at 25~30ms/req.

It seems to me that not much can be improve on the signature computation itself, though the rsa library mentioned above was last updated almost 1.5 years ago. I wonder if there are other techniques or designs that may optimize the performance of this process to achieve higher scalability. Or do we simply have to throw in more hardware and try to solve it in a brute force way?

One optimization can be making the API endpoint accept multiple file signings per request and return the signed URLs in bulk rather than dealing with them individually in separate requests, but the total time necessary for computing all those signatures is still there.

Answer 1

Use Signed Cookies

When I use CloudFront with many private URLs, I prefer to use Signed Cookies when all the restrictions are met. This does not speed up the generation of signed cookies but it reduces the number of signing requests to be one per user until they expire.

Tuning RSA Signature Generation

I can imagine you may have requirements which render signed cookies as an invalid option. In that case I tried to speed up the signing by comparing the RSA module used with boto and cryptography. Two additional alternative options are m2crypto and pycrypto but for this example I will use cryptography.

In order to test performance of signing URLs with different modules I reduced the method _sign_string to remove any logic except the signing of a string then created a new Distribution class. Then I took the private key and example URL from boto tests to test with.

The results show that cryptography is quicker but still requires close to 1ms per signing request. These results are skewed higher by iPython's use of scoped variables in timing.

timeit -n10000 rsa_distribution.create_signed_url(url, message, expire_time)
10000 loops, best of 3: 6.01 ms per loop

timeit -n10000 cryptography_distribution.create_signed_url(url, message, expire_time)
10000 loops, best of 3: 644 µs per loop

The full script:

from cryptography.hazmat.primitives.asymmetric import padding
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives import hashes

import rsa

from boto.cloudfront.distribution import Distribution

from textwrap import dedent

# The private key provided in the Boto tests
pk_key = dedent("""
    -----BEGIN RSA PRIVATE KEY-----
    MIICXQIBAAKBgQDA7ki9gI/lRygIoOjV1yymgx6FYFlzJ+z1ATMaLo57nL57AavW
    hb68HYY8EA0GJU9xQdMVaHBogF3eiCWYXSUZCWM/+M5+ZcdQraRRScucmn6g4EvY
    2K4W2pxbqH8vmUikPxir41EeBPLjMOzKvbzzQy9e/zzIQVREKSp/7y1mywIDAQAB
    AoGABc7mp7XYHynuPZxChjWNJZIq+A73gm0ASDv6At7F8Vi9r0xUlQe/v0AQS3yc
    N8QlyR4XMbzMLYk3yjxFDXo4ZKQtOGzLGteCU2srANiLv26/imXA8FVidZftTAtL
    viWQZBVPTeYIA69ATUYPEq0a5u5wjGyUOij9OWyuy01mbPkCQQDluYoNpPOekQ0Z
    WrPgJ5rxc8f6zG37ZVoDBiexqtVShIF5W3xYuWhW5kYb0hliYfkq15cS7t9m95h3
    1QJf/xI/AkEA1v9l/WN1a1N3rOK4VGoCokx7kR2SyTMSbZgF9IWJNOugR/WZw7HT
    njipO3c9dy1Ms9pUKwUF46d7049ck8HwdQJARgrSKuLWXMyBH+/l1Dx/I4tXuAJI
    rlPyo+VmiOc7b5NzHptkSHEPfR9s1OK0VqjknclqCJ3Ig86OMEtEFBzjZQJBAKYz
    470hcPkaGk7tKYAgP48FvxRsnzeooptURW5E+M+PQ2W9iDPPOX9739+Xi02hGEWF
    B0IGbQoTRFdE4VVcPK0CQQCeS84lODlC0Y2BZv2JxW3Osv/WkUQ4dslfAQl1T303
    7uwwr7XTroMv8dIFQIPreoPhRKmd/SbJzbiKfS/4QDhU
    -----END RSA PRIVATE KEY-----""")

# Initializing keys in a global context
cryptography_private_key = serialization.load_pem_private_key(
    pk_key,
    password=None,
    backend=default_backend())


# Instantiate a signer object using PKCS 1v 15, this is not recommended but required for Amazon
def sign_with_cryptography(message):
    signer = cryptography_private_key.signer(
        padding.PKCS1v15(),
        hashes.SHA1())

    signer.update(message)
    return signer.finalize()


# Initializing the key in a global context
rsa_private_key = rsa.PrivateKey.load_pkcs1(pk_key)


def sign_with_rsa(message):
    signature = rsa.sign(str(message), rsa_private_key, 'SHA-1')

    return signature


# All this information comes from the Boto tests.
url = "http://d604721fxaaqy9.cloudfront.net/horizon.jpg?large=yes&license=yes"
expected_url = "http://d604721fxaaqy9.cloudfront.net/horizon.jpg?large=yes&license=yes&Expires=1258237200&Signature=Nql641NHEUkUaXQHZINK1FZ~SYeUSoBJMxjdgqrzIdzV2gyEXPDNv0pYdWJkflDKJ3xIu7lbwRpSkG98NBlgPi4ZJpRRnVX4kXAJK6tdNx6FucDB7OVqzcxkxHsGFd8VCG1BkC-Afh9~lOCMIYHIaiOB6~5jt9w2EOwi6sIIqrg_&Key-Pair-Id=PK123456789754"
message = "PK123456789754"
expire_time = 1258237200


class CryptographyDistribution(Distribution):
    def _sign_string(
            self,
            message,
            private_key_file=None,
            private_key_string=None):
        return sign_with_cryptography(message)


class RSADistribution(Distribution):
    def _sign_string(
            self,
            message,
            private_key_file=None,
            private_key_string=None):
        return sign_with_rsa(message)


cryptography_distribution = CryptographyDistribution()
rsa_distribution = RSADistribution()

cryptography_url = cryptography_distribution.create_signed_url(
    url,
    message,
    expire_time)

rsa_url = rsa_distribution.create_signed_url(
    url,
    message,
    expire_time)

assert cryptography_url == rsa_url == expected_url, "URLs do not match"

Conclusion

Although the cryptography module performs better in this test, I recommend trying to find a way to utilize signed cookies but I hope this information is useful.

Answer 2

Briefly

Consider whether you can (in addition to using python-cryptography, per @erik-e) use a shorter key length (and probably change keys more frequently), given the particulars of your use case. While I can sign with the 2048-bit key AWS generated in ~1550µs, it only takes ~307µs at 1028 bits, ~184µs at 768 bits, and ~113µs at 512 bits.

Explanation

After looking into this for a bit, I'm going to go in another direction and build off of the (already great) answer @erik-e gave. I should mention before I get into it that I don't know how acceptable this idea is; I am just reporting on the performance impact it has (see the end of the post for a question I asked on the security SE seeking input on this).

I was collecting timings on signing with cryptography as @erik-e suggests, and because of the still large performance gulf between it and our existing signing method for S3, I decided to profile the code to see if it looked like there might be anything obvious chewing up time:

>>> cProfile.runctx('[sign_url_cloudfront2("...") for x in range(0,100)]', globals(), locals(), sort="time")
         9403 function calls in 0.218 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
      200    0.161    0.001    0.161    0.001 {_Cryptography_cffi_a269d620xd5c405b7.EVP_PKEY_sign}
      100    0.006    0.000    0.186    0.002 rsa.py:214(_finalize_pkey_ctx)
     1200    0.004    0.000    0.008    0.000 {isinstance}
      400    0.004    0.000    0.007    0.000 api.py:212(new)
      100    0.003    0.000    0.218    0.002 views.py:888(sign_url_cloudfront2)
      300    0.002    0.000    0.004    0.000 abc.py:128(__instancecheck__)
      100    0.002    0.000    0.008    0.000 hashes.py:53(finalize)
      200    0.002    0.000    0.005    0.000 gc_weakref.py:10(build)
      100    0.002    0.000    0.007    0.000 hashes.py:15(__init__)
      100    0.002    0.000    0.018    0.000 rsa.py:151(__init__)
      100    0.002    0.000    0.014    0.000 hashes.py:68(__init__)
      200    0.002    0.000    0.003    0.000 gc_weakref.py:14(remove)
      200    0.002    0.000    0.003    0.000 api.py:239(cast)
      100    0.002    0.000    0.190    0.002 rsa.py:207(finalize)
      200    0.001    0.000    0.007    0.000 api.py:325(gc)
      500    0.001    0.000    0.001    0.000 {getattr}
      400    0.001    0.000    0.001    0.000 {_cffi_backend.newp}
      400    0.001    0.000    0.001    0.000 api.py:150(_typeof)
      200    0.001    0.000    0.002    0.000 api.py:266(buffer)
      200    0.001    0.000    0.001    0.000 utils.py:18(<lambda>)
      300    0.001    0.000    0.001    0.000 _weakrefset.py:68(__contains__)
      200    0.001    0.000    0.001    0.000 {_cffi_backend.buffer}
      100    0.001    0.000    0.002    0.000 hashes.py:49(update)
      100    0.001    0.000    0.010    0.000 hashes.py:102(finalize)
      100    0.001    0.000    0.003    0.000 hashes.py:88(update)
      200    0.001    0.000    0.001    0.000 {method 'encode' of 'str' objects}
      100    0.001    0.000    0.019    0.000 rsa.py:528(signer)
      300    0.001    0.000    0.001    0.000 {len}
      100    0.001    0.000    0.001    0.000 base64.py:42(b64encode)
      100    0.001    0.000    0.008    0.000 backend.py:148(create_hash_ctx)
      200    0.001    0.000    0.001    0.000 {_cffi_backend.cast}
      200    0.001    0.000    0.001    0.000 {_Cryptography_cffi_a269d620xd5c405b7.EVP_get_digestbyname}
      100    0.001    0.000    0.001    0.000 {method 'format' of 'str' objects}
      100    0.001    0.000    0.003    0.000 rsa.py:204(update)
      200    0.000    0.000    0.000    0.000 {method 'pop' of 'dict' objects}
      100    0.000    0.000    0.000    0.000 {binascii.b2a_base64}
      200    0.000    0.000    0.000    0.000 {_cffi_backend.typeof}
      100    0.000    0.000    0.000    0.000 {time.time}
      100    0.000    0.000    0.000    0.000 {_Cryptography_cffi_a269d620xd5c405b7.EVP_DigestUpdate}
        1    0.000    0.000    0.218    0.218 <string>:1(<module>)
      100    0.000    0.000    0.000    0.000 {_Cryptography_cffi_a269d620xd5c405b7.EVP_DigestInit_ex}
      100    0.000    0.000    0.000    0.000 {_Cryptography_cffi_a269d620xd5c405b7.EVP_PKEY_CTX_new}
      100    0.000    0.000    0.000    0.000 {_Cryptography_cffi_a269d620xd5c405b7.EVP_PKEY_CTX_free}
      100    0.000    0.000    0.000    0.000 {_Cryptography_cffi_a269d620xd5c405b7.EVP_DigestFinal_ex}
      100    0.000    0.000    0.000    0.000 {_Cryptography_cffi_a269d620xd5c405b7.EVP_MD_CTX_create}
      100    0.000    0.000    0.000    0.000 {_Cryptography_cffi_a269d620xd5c405b7.EVP_PKEY_CTX_set_rsa_padding}
      100    0.000    0.000    0.000    0.000 {_Cryptography_cffi_a269d620xd5c405b7.EVP_PKEY_size}
      100    0.000    0.000    0.000    0.000 {method 'translate' of 'str' objects}
      100    0.000    0.000    0.000    0.000 {_Cryptography_cffi_a269d620xd5c405b7.EVP_MD_CTX_cleanup}
      100    0.000    0.000    0.000    0.000 {_Cryptography_cffi_a269d620xd5c405b7.EVP_PKEY_CTX_set_signature_md}
      100    0.000    0.000    0.000    0.000 {_Cryptography_cffi_a269d620xd5c405b7.EVP_PKEY_sign_init}
      100    0.000    0.000    0.000    0.000 {_Cryptography_cffi_a269d620xd5c405b7.EVP_MD_CTX_destroy}
        1    0.000    0.000    0.000    0.000 {range}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}

While there might be some small savings lurking inside signer, the vast majority of the time is spent inside of the finalize() call, and almost all of that time is spent inside the actual sign call to openssl. While this was a little disappointing, it was a clear indicator that I should look to the actual signing process for savings.

I was just using the 2048-bit key CloudFront generated for us, so I decided to see what impact a smaller key would have on performance. I re-ran the profile using the shorter key:

>>> cProfile.runctx('[sign_url_cloudfront2("...") for x in range(0,100)]', globals(), locals(), sort="time")
        9203 function calls in 0.063 seconds

  Ordered by: internal time

  ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     100    0.008    0.000    0.008    0.000 {_Cryptography_cffi_a269d620xd5c405b7.EVP_PKEY_sign}
     400    0.005    0.000    0.008    0.000 api.py:212(new)
     100    0.004    0.000    0.033    0.000 rsa.py:214(_finalize_pkey_ctx)
    1200    0.004    0.000    0.008    0.000 {isinstance}
     100    0.003    0.000    0.063    0.001 views.py:897(sign_url_cloudfront2)
     300    0.002    0.000    0.004    0.000 abc.py:128(__instancecheck__)
     100    0.002    0.000    0.008    0.000 hashes.py:53(finalize)
     200    0.002    0.000    0.005    0.000 gc_weakref.py:10(build)
     100    0.002    0.000    0.007    0.000 hashes.py:15(__init__)
     100    0.002    0.000    0.014    0.000 hashes.py:68(__init__)
     100    0.002    0.000    0.018    0.000 rsa.py:151(__init__)
     200    0.002    0.000    0.003    0.000 gc_weakref.py:14(remove)
     100    0.001    0.000    0.036    0.000 rsa.py:207(finalize)
     200    0.001    0.000    0.003    0.000 api.py:239(cast)
     200    0.001    0.000    0.006    0.000 api.py:325(gc)
     500    0.001    0.000    0.001    0.000 {getattr}
     200    0.001    0.000    0.002    0.000 api.py:266(buffer)
     400    0.001    0.000    0.001    0.000 {_cffi_backend.newp}
     400    0.001    0.000    0.001    0.000 api.py:150(_typeof)
     100    0.001    0.000    0.010    0.000 hashes.py:102(finalize)
     200    0.001    0.000    0.002    0.000 utils.py:18(<lambda>)
     300    0.001    0.000    0.001    0.000 _weakrefset.py:68(__contains__)
     100    0.001    0.000    0.002    0.000 hashes.py:88(update)
     100    0.001    0.000    0.001    0.000 hashes.py:49(update)
     200    0.001    0.000    0.001    0.000 {method 'encode' of 'str' objects}
     200    0.001    0.000    0.001    0.000 {_Cryptography_cffi_a269d620xd5c405b7.EVP_get_digestbyname}
     100    0.001    0.000    0.001    0.000 base64.py:42(b64encode)
     100    0.001    0.000    0.008    0.000 backend.py:148(create_hash_ctx)
     100    0.001    0.000    0.019    0.000 rsa.py:520(signer)
     200    0.001    0.000    0.001    0.000 {_cffi_backend.buffer}
     200    0.001    0.000    0.001    0.000 {method 'pop' of 'dict' objects}
     200    0.001    0.000    0.001    0.000 {_cffi_backend.cast}
     100    0.001    0.000    0.001    0.000 {method 'format' of 'str' objects}
     100    0.001    0.000    0.001    0.000 {time.time}
     100    0.001    0.000    0.003    0.000 rsa.py:204(update)
     200    0.000    0.000    0.000    0.000 {len}
     200    0.000    0.000    0.000    0.000 {_cffi_backend.typeof}
     100    0.000    0.000    0.000    0.000 {binascii.b2a_base64}
     100    0.000    0.000    0.000    0.000 {method 'translate' of 'str' objects}
       1    0.000    0.000    0.063    0.063 <string>:1(<module>)
     100    0.000    0.000    0.000    0.000 {_Cryptography_cffi_a269d620xd5c405b7.EVP_DigestUpdate}
     100    0.000    0.000    0.000    0.000 {_Cryptography_cffi_a269d620xd5c405b7.EVP_PKEY_CTX_new}
     100    0.000    0.000    0.000    0.000 {_Cryptography_cffi_a269d620xd5c405b7.EVP_DigestInit_ex}
     100    0.000    0.000    0.000    0.000 {_Cryptography_cffi_a269d620xd5c405b7.EVP_MD_CTX_destroy}
     100    0.000    0.000    0.000    0.000 {_Cryptography_cffi_a269d620xd5c405b7.EVP_DigestFinal_ex}
     100    0.000    0.000    0.000    0.000 {_Cryptography_cffi_a269d620xd5c405b7.EVP_MD_CTX_create}
     100    0.000    0.000    0.000    0.000 {_Cryptography_cffi_a269d620xd5c405b7.EVP_PKEY_sign_init}
     100    0.000    0.000    0.000    0.000 {_Cryptography_cffi_a269d620xd5c405b7.EVP_PKEY_size}
     100    0.000    0.000    0.000    0.000 {_Cryptography_cffi_a269d620xd5c405b7.EVP_MD_CTX_cleanup}
     100    0.000    0.000    0.000    0.000 {_Cryptography_cffi_a269d620xd5c405b7.EVP_PKEY_CTX_free}
     100    0.000    0.000    0.000    0.000 {_Cryptography_cffi_a269d620xd5c405b7.EVP_PKEY_CTX_set_signature_md}
     100    0.000    0.000    0.000    0.000 {_Cryptography_cffi_a269d620xd5c405b7.EVP_PKEY_CTX_set_rsa_padding}
       1    0.000    0.000    0.000    0.000 {range}
       1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}

As mentioned in my comment on erik-e's answer, the runtime I saw for our full signing method using the 2048-bit key with the cryptography module was ~1550µs. Repeating this same test with the 512-bit key brings the runtime down to about ~113µs (a stone's-throw from the ~30µs of our S3 signing method).

This result seems meaningful, but it hinges on how acceptable it is to use a shorter key for your purpose. I was able to find a comment from March on a Mozilla issue report suggesting a 512-bit key could be broken for $75 in 8 hours on EC2.