Python Generated Signature for S3 Post

Question

I think I've read nearly everything there is to read on base-64 encoding of a signature for in-browser, form-based post to S3: old docs and new docs. For instance:

http://doc.s3.amazonaws.com/proposals/post.html

And even found this:

http://s3.amazonaws.com/doc/s3-example-code/post/post_sample.html

Rather than using the above or Amazon's newer policy generator, or fiddle around with Boto, I'm trying to draft a simpler .py script that pulls the policy JSON from a plaintext file (policy.txt), and then generates the necessary base-64 encoded signature to help me draft the HTML form.

The signature itself (which is reliant on the encoded policy) is NOT being encoded correctly...maybe due to some sort of utf-8 vs. ascii or \n (newline) issue?

The script I'm working with is below, the policy and the AWS Secret Key private_key are from an AWS test case I'm using to see if this script works. The correctly encoded signature--as quoted by Amazon--is included in the script below for reference.

Can anyone tell me why the signature as calculated below does not match the reference signature provided by Amazon?:

In other words:

Why this is correctly encoded:

policy_encoded = base64.b64encode(policy)

but this one is NOT:

signature = base64.b64encode(hmac.new(private_key, policy_encoded, sha).digest())

PYTHON signature calculator...

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import base64, hmac, sha
from sys import argv

script, policy = argv

private_key = 'uV3F3YluFJax1cknvbcGwgjvx4QpvB+leU8dUj2o'
input = open("..Desktop/policy.txt", "rb")
policy = input.read()
policy_encoded = base64.b64encode(policy)
signature = base64.b64encode(hmac.new(private_key, policy_encoded, sha).digest())
print "Your policy base-64 encoded is %s." % (policy_encoded)
print "Your signature base-64 encoded is %s." % (signature)
print "Your signature encoded should be 2qCp0odXe7A9IYyUVqn0w2adtCA="

JSON Policy (policy.txt--UTF-8)

{ "expiration": "2007-12-01T12:00:00.000Z",
"conditions": [
{"bucket": "johnsmith"},
["starts-with", "$key", "user/eric/"],
{"acl": "public-read"},
{"success_action_redirect": "http://johnsmith.s3.amazonaws.com/successful_upload.html"},
["starts-with", "$Content-Type", "image/"],
{"x-amz-meta-uuid": "14365123651274"},
["starts-with", "$x-amz-meta-tag", ""]
]
}

Answer 1

I think this is down to the contents of your policy.txt file.

I took the policy from the referenced link (http://doc.s3.amazonaws.com/proposals/post.html) and saved it as policy.txt

{ "expiration": "2007-12-01T12:00:00.000Z",
  "conditions": [
    {"bucket": "johnsmith" },
    ["starts-with", "$key", "user/eric/"],
    {"acl": "public-read" },
    {"redirect": "http://johnsmith.s3.amazonaws.com/successful_upload.html" },
    ["starts-with", "$Content-Type", "image/"],
    {"x-amz-meta-uuid": "14365123651274"},
    ["starts-with", "$x-amz-meta-tag", ""],
  ]
}

In order to get the exact same signature, this file must have the exact same contents.

For reference, when I copied and pasted: MD5 (policy.txt) = 5bce89d9ff799e2064c136d76bc7fc7a

If I use the following script (same as yours, just adjust filename and remove args)

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import base64, hmac, sha

private_key = 'uV3F3YluFJax1cknvbcGwgjvx4QpvB+leU8dUj2o'
input = open("policy.txt", "rb")
policy = input.read()
policy_encoded = base64.b64encode(policy)
signature = base64.b64encode(hmac.new(private_key, policy_encoded, sha).digest())
print "Your policy base-64 encoded is %s." % (policy_encoded)
print "Your signature base-64 encoded is %s." % (signature)
print "Your signature encoded should be 2qCp0odXe7A9IYyUVqn0w2adtCA="

Output I get:

Your policy base-64 encoded is
eyAiZXhwaXJhdGlvbiI6ICIyMDA3LTEyLTAxVDEyOjAwOjAwLjAwMFoiLAogICJjb25kaXRpb25zIjo
gWwogICAgeyJidWNrZXQiOiAiam9obnNtaXRoIiB9LAogICAgWyJzdGFydHMtd2l0aCIsICIka2V5Ii
wgInVzZXIvZXJpYy8iXSwKICAgIHsiYWNsIjogInB1YmxpYy1yZWFkIiB9LAogICAgeyJyZWRpcmVjd
CI6ICJodHRwOi8vam9obnNtaXRoLnMzLmFtYXpvbmF3cy5jb20vc3VjY2Vzc2Z1bF91cGxvYWQuaHRt
bCIgfSwKICAgIFsic3RhcnRzLXdpdGgiLCAiJENvbnRlbnQtVHlwZSIsICJpbWFnZS8iXSwKICAgIHs
ieC1hbXotbWV0YS11dWlkIjogIjE0MzY1MTIzNjUxMjc0In0sCiAgICBbInN0YXJ0cy13aXRoIiwgIi
R4LWFtei1tZXRhLXRhZyIsICIiXSwKICBdCn0K
Your signature base-64 encoded is 2qCp0odXe7A9IYyUVqn0w2adtCA=
Your signature encoded should be 2qCp0odXe7A9IYyUVqn0w2adtCA=

So, your code works, I just think you're signing a slightly different policy (whitespace differences)