Amazon S3 Policy Signing in Java

Question

For some reason, I'm struggling with the signature generation for my Amazon S3 upload policy. I swear I had this working at one point but no longer. Any help would be much appreciated. I need a fresh set of eyes.

When comparing to the output from Amazon S3 Signature Tester, I am not getting the same signature. However, when I directly use the signature coming out of that tool, everything works fine. So the issue is definitely in my signing process. Also, the "String to be signed" hex-decoded coming out of that tool is identical to my input policy being signed.

The AWS docs say the process for constructing a policy signature should go like this:

1. Encode the policy using UTF-8.
2. Encode those UTF-8 bytes using Base64.
3. Sign the policy with your Secret Access Key using HMAC SHA-1.
4. Encode the SHA-1 signature using Base64.

Seems straight-forward enough. The only place for ambiguity might be in #3. The AWS docs show a sample snippet for generating HMAC-SHA1 and this is consistent with other Java cryptography examples I've seen.

I'm using v1.6 of Apache Commons implementation of Base64. My signing code basically looks like this:

import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
import org.apache.commons.codec.binary.Base64;

/* ... */

private static final String UTF8 = "UTF-8";
private static final String HMACSHA1 = "HmacSHA1";

public static String sign(String secret, String data) {
    byte[] dataBytes = data.getBytes(UTF8);
    byte[] secretBytes = secret.getBytes(UTF8);

    SecretKeySpec signingKey = new SecretKeySpec(secretBytes, HMACSHA1);

    Mac mac = Mac.getInstance(HMACSHA1);
    mac.init(signingKey);
    byte[] signature = mac.doFinal(dataBytes);

    return Base64.encodeBase64String(signature);
}

And then my usage of this signing looks like:

String signature = sign(
    /* AWS Secret Access Key copied directly out of the AWS Console */,
    /* policy properly serialized as JSON */);

Answer 1

Okay, I found it. Apparently today I've been effectively skipping step #2. I did encode the policy JSON as Base64 but then I am directly signing the JSON string not the Base64 string.

Step #3 should probably be reworded to "Sign the Base64 policy with your Secret Access Key using HMAC SHA-1."

I guess I'll leave this up in case anyone else comes across a similar issue.

Answer 2

Now, this procedure is officially supported. http://aws.amazon.com/articles/1434

import sun.misc.BASE64Encoder;
import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;

String policy = (new BASE64Encoder()).encode(
policy_document.getBytes("UTF-8")).replaceAll("\n","").replaceAll("\r","");

Mac hmac = Mac.getInstance("HmacSHA1");
hmac.init(new SecretKeySpec(
aws_secret_key.getBytes("UTF-8"), "HmacSHA1"));
String signature = (new BASE64Encoder()).encode(
hmac.doFinal(policy.getBytes("UTF-8")))
.replaceAll("\n", "");

*Beware the window implementation for this example as some found problem from the comments of the post and solution to the problem was also provided there.

The result can be verified by this http://s3.amazonaws.com/doc/s3-example-code/post/post_sample.html

However, some said this "org.apache.commons.codec.binary.Base64" is better because of this. http://www.asgarli.net/2011/03/replacing-sunmiscbase64encoder-and.html