SignatureDoesNotMatch - Boto3 Django-storages

Question

I have the following config:

Django/DRF Boto3 Django-storages

I am using an IAM user credentials with one set of keys. I have removed all other sets of keys including root keys from my account, to eliminate keys mismatch.

I created a new bucket my-prod-bucket. Updated the bucket name settings in my env file. I ran python3 manage.py collectstatic and it created the new bucket without a problem.

my .env:

AWS_ACCESS_KEY_ID=something
AWS_SECRET_ACCESS_KEY=something
AWS_STORAGE_BUCKET_NAME=my-prod-bucket

my settings.py (using python-decouple to grab from .env):

AWS_ACCESS_KEY_ID = config('AWS_ACCESS_KEY_ID')
AWS_SECRET_ACCESS_KEY = config('AWS_SECRET_ACCESS_KEY')
AWS_STORAGE_BUCKET_NAME = config('AWS_STORAGE_BUCKET_NAME')

AWS_S3_CUSTOM_DOMAIN = '%s.s3.ca-central-1.amazonaws.com' % AWS_STORAGE_BUCKET_NAME
AWS_S3_REGION_NAME = 'ca-central-1'


AWS_HEADERS = {
    'CacheControl': 'max-age=86400',
}

AWS_STATIC_LOCATION = 'static'
STATIC_URL = 'https://%s/%s/' % (AWS_S3_CUSTOM_DOMAIN, AWS_STATIC_LOCATION)
STATICFILES_STORAGE = 'portal.storage_backends.StaticStorage'

# =======
AWS_DEFAULT_ACL = None

AWS_AUTO_CREATE_BUCKET = True
S3_USE_SIGV4 = True

I can upload and delete however when I try to download a file I get:

<Error>
<Code>SignatureDoesNotMatch</Code>
<Message>
The request signature we calculated does not match the signature you provided. Check your key and signing method.
</Message>
<AWSAccessKeyId>AKIA6FUWELHP36HW6QOT</AWSAccessKeyId>
<StringToSign>
AWS4-HMAC-SHA256 20200211T215631Z 20200211/ca-central-1/s3/aws4_request 703b799a80d9efd9f9e06a01ab30a8a721f2a9bafe6a3d5c92b045ea769b0d87
</StringToSign>
<SignatureProvided>
46bd882624f966d9cb8914d279f7c8f91a2b3e5e577525c13069e29f8891c1ee
</SignatureProvided>
<StringToSignBytes>
41 57 53 34 2d 48 4d 41 43 2d 53 48 41 32 35 36 0a 32 30 32 30 30 32 31 31 54 32 31 35 36 33 31 5a 0a 32 30 32 30 30 32 31 31 2f 63 61 2d 63 65 6e 74 
</StringToSignBytes>
<CanonicalRequest>
GET /media/private/cities/20/2017/london_2016.csv X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIA6FUWELHP36%2F20200211%2Fca-central-1%2Fs3%2Faws4_request&X-Amz-Date=20200211T215631Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host host:my-prod-bucket.s3.ca-central-1.amazonaws.com host UNSIGNED-PAYLOAD
</CanonicalRequest>
<CanonicalRequestBytes>
47 45 54 0a 2f 6d 65 64 69 61 2f 70 72 69 76 61 74 65 2f 63 69 74 69 65 73 2f 32 30 2f 32 30 31 37 2f 45 43 35 2e 31 2f 6c 6f 6e 64 6f 6e 5f 32 30 31 36 2e 63 73 76 0a 58 2d 41 6d 7a 2d 41 6c 67 6f 72 69 74 68 6d 3d 41 57 53 
</CanonicalRequestBytes>
<RequestId>6A85C2780914C0F5</RequestId>
<HostId>
WtPC4cEV60ybq2pEdfghdfg23tg123lVV6l/iHiaSAjL4DS0=
</HostId>
</Error>

I'm not sure what I'm doing wrong. I searched every post on this error but couldn't find anything recent that fits my scenario.

Any ideas on how to troubleshoot will be greatly appreciated.

Answer 1

Ok, so after spending nearly 2 days trying to make sense of this, this is what I came up with:

The problem in my case was that the bucket created in zone ca-central-1. Once I changed the request to a bucket in us-east-1 everything was immediately working fine without that error. Everything on my end was set perfectly.

Now, the next day I tried to connect to that same ca-central-1 bucket again and this time it worked. No signature mismatch error.

At this point I'm thinking maybe there's a 'time-delay' on AWS S3 when creating buckets in some areas until they function properly.

To test my theory, I created a new bucket in ca-central-1 and tried to connect to it. Again, same error as above for the new bucket. Waited till the next day, tried again - and everything was working fine.

Keep the 'time-delay' (for a lack of a better explanation) in mind if ever encountering the same issue.