AWS SignatureDoesNotMatch error but signing code seems valid as well as key pair

Question

and thanks for looking.

I am trying to write my own lightweight PHP class to generate AWS authentication headers. From what I can tell it is functioning correctly. I have tested it's output with the examples provided in the AWS documentation here: http://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-header-based-auth.html

and my code produces exactly the output as given for each example, so I am fairly confident that the signature generation is correct.

I have also tested the key and secret with the AWS PHP SDK and proven it works with this code: (keys obviously obfuscated :) )

require 'vendor/autoload.php';
use Aws\S3\S3Client;

$client = S3Client::factory(array(
   'key' => 'xxxxxxxxxxxxxxxxxxxx',
   'secret' => 'ssssssssssssssssssssssssssssssssssssssss',
   'scheme' => 'http'
));

$result = $client->listBuckets();

foreach ($result['Buckets'] as $bucket) {
    // Each Bucket value will contain a Name and CreationDate
    echo "{$bucket['Name']} - {$bucket['CreationDate']}\n";
}

(Just testing so key is written in the code which I know is not correct best practice :), and switched to HTTP so I could inspect the output with wireshark)

The wireshark output showed that the headers included from the SDK don't match up with those provided in the documentation I have linked earlier! There seems to be a lot of conflicting documentation on the AWS docs site.

As my signature generation code matches the examples, I am suspicious that perhaps I have something wrong when using curl to send the request? I have tried POST and GET as the methods with no difference in response. The String to Sign in the error message and the Canonical Request both match those produced by the signing functions.

My code snippet is:

printf("URL: $e\n");
if ($ch = curl_init("http://s3-eu-west-1.amazonaws.com".$e)){ // Create curl request object

    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

    $curlHeaders = array();
    foreach ($awsobj->headers as $key => $val){
        $curlHeaders[] = $key.': '.$val;
    }
    $curlHeaders[] = 'Authorization: '.$ah['Authorization'];

    curl_setopt($ch, CURLOPT_HTTPHEADER, $curlHeaders);

    printf("CREQ:\n%s\n---", $awsobj->getCREQ());
    printf("STS:\n%s\n---", $awsobj->getSTS());

    $res = curl_exec($ch);

    var_dump($res);
}

URL it is calling against is: http://s3-eu-west-1.amazonaws.com/?LocationConstraint=eu-west-1

With String to Sign of:

AWS4-HMAC-SHA256
Sun, 29 Nov 2015 10:57:02 +0000
20151129/eu-west-1/s3/aws4_request
2b1435293edc751d0d80efc9016433a2635de23bc8c0d2e97d3a54c0cfadd74b

Canonical Request of:

GET
/
LocationConstraint=eu-west-1
date:Sun, 29 Nov 2015 10:57:02 +0000
host:s3-eu-west-1.amazonaws.com
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

date;host;x-amz-content-sha256
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

(There is no return at the end of these)

Response from S3 is :

<Error><Code>SignatureDoesNotMatch</Code>
<Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message>   

<StringToSign>AWS4-HMAC-SHA256
Sun, 29 Nov 2015 10:57:02 +0000
20151129/eu-west-1/s3/aws4_request
2b1435293edc751d0d80efc9016433a2635de23bc8c0d2e97d3a54c0cfadd74b</StringToSign>
<StringToSignBytes>41 57 53 34 2d 48 4d 41 43 2d 53 48 41 32 35 36 0a 53 75 6e 2c 20 32 39 20 4e 6f 76 20 32 30 31 35 20 31 30 3a 35 37 3a 30 32 20 2b 30 30 30 30 0a 32 30 31 35 31 31 32 39 2f 65 75 2d 77 65 73 74 2d 31 2f 73 33 2f 61 77 73 34 5f 72 65 71 75 65 73 74 0a 32 62 31 34 33 35 32 39 33 65 64 63 37 35 31 64 30 64 38 30 65 66 63 39 30 31 36 34 33 33 61 32 36 33 35 64 65 32 33 62 63 38 63 30 64 32 65 39 37 64 33 61 35 34 63 30 63 66 61 64 64 37 34 62</StringToSignBytes>
<CanonicalRequest>GET
/
LocationConstraint=eu-west-1
date:Sun, 29 Nov 2015 10:57:02 +0000
host:s3-eu-west-1.amazonaws.com
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

date;host;x-amz-content-sha256
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855</CanonicalRequest>
<CanonicalRequestBytes>47 45 54 0a 2f 0a 4c 6f 63 61 74 69 6f 6e 43 6f 6e 73 74 72 61 69 6e 74 3d 65 75 2d 77 65 73 74 2d 31 0a 64 61 74 65 3a 53 75 6e 2c 20 32 39 20 4e 6f 76 20 32 30 31 35 20 31 30 3a 35 37 3a 30 32 20 2b 30 30 30 30 0a 68 6f 73 74 3a 73 33 2d 65 75 2d 77 65 73 74 2d 31 2e 61 6d 61 7a 6f 6e 61 77 73 2e 63 6f 6d 0a 78 2d 61 6d 7a 2d 63 6f 6e 74 65 6e 74 2d 73 68 61 32 35 36 3a 65 33 62 30 63 34 34 32 39 38 66 63 31 63 31 34 39 61 66 62 66 34 63 38 39 39 36 66 62 39 32 34 32 37 61 65 34 31 65 34 36 34 39 62 39 33 34 63 61 34 39 35 39 39 31 62 37 38 35 32 62 38 35 35 0a 0a 64 61 74 65 3b 68 6f 73 74 3b 78 2d 61 6d 7a 2d 63 6f 6e 74 65 6e 74 2d 73 68 61 32 35 36 0a 65 33 62 30 63 34 34 32 39 38 66 63 31 63 31 34 39 61 66 62 66 34 63 38 39 39 36 66 62 39 32 34 32 37 61 65 34 31 65 34 36 34 39 62 39 33 34 63 61 34 39 35 39 39 31 62 37 38 35 32 62 38 35 35</CanonicalRequestBytes>
<RequestId>6C049EA89D65D27E</RequestId>
<HostId>HSGO1Iz6LIT5jkHHGM/XCI0ElnSDiQseb1CUcg4RxXjb+xplWoVCFbgJoT6CPvpCsoOSIS7m7VI=</HostId>

(Trimmed out Access Key)

I have spent the last few days googling and tweaking my code. I cannot spot anything wrong with the signature generation, as you can see my generated STS and CREQ both match those produced by S3. My code passes and matches all the examples on the link above, including the Authentication header credentials.

I suspect I am

• either missing something that S3 wants me to include (required header, I have searched for these but found none, the output is the same with or without LocationConstraint, and also if I point it at us-east-1).
• Mangling curl some how
• using the wrong kind of authentication header (as I said the request generated by the SDK looks completely different but also doesn't adhere to what the docs say is required).

As Per Rhythmic Fistman's suggestion I have also tried replacing the Date header with an x-amz-date header, but this didn't resolve the issue. The format of my new header was :

x-amz-date:20151130T143334Z

Created on 30th nov 2015 at 14:33

Thanks

Answer 1

I have spotted my error :/ it is not to do with curl but to do with how I approached the signing.

I have been testing my code by way of testing the output at each stage of the process and comparing the results with those given by the Amazon examples. These all passed and so I was certain my code was correct.

However, the generation of the String to Sign requires a hash of the Canonical request. In my code I have a function that generates the canonical request and outputs it, but it also generates a hash of it and stores this in a public variable in the object. My string to sign generating function uses this variable in the generation of the string to sign.

Now as my test code called the canonical request, tested it's output was valid, then called the string to sign and tested it's result was valid, it was inadvertently causing the hash to be created.

In my actual code that calls the API, there was no need to output the actual canonical request, therefore the hash was never created and the string to sign was invalid. The output I added whilst trying to diagnose the problem was added towards the end of the code, after the signature was generated.

On realising my mistake I have just moved the call to output the canonical request before the call to generate the signature and hey presto it works!

ARGH!

Thought it best to explain the error of my ways here, I admit it's my own daft fault, and that if writing code to be able to view stages of a process, ensure they don't then depend on those stages, or exposure methods being called to function correctly.