Amazon AWS Cognito and Python Boto3 to establish AWS connection and upload file to Bucket

Question

I'm trying to use the AWS cognito service to authenticate and upload a file. I have been provided my regionType, identityPool, AWS account ID, and UnAuthRole. I also know the production and development bucket names.

I think I am setting the AWS Access Key and AWS Secret Key... I want to authenticate with cognito and use the results to allow me to do an bucket listing and later a file upload.

What am I doing wrong? How can I use the cognito id to establish an S3 connection?

Here is my code and the resulting error:

#!/usr/bin/python

import boto3
import boto
#boto.set_stream_logger('foo')
import json
client = boto3.client('cognito-identity','us-east-1')
resp =  client.get_id(AccountId='<ACCNTID>',IdentityPoolId='<IDPOOLID>')
print "\nIdentity ID: %s"%(resp['IdentityId'])
print "\nRequest ID: %s"%(resp['ResponseMetadata']['RequestId'])
resp = client.get_open_id_token(IdentityId=resp['IdentityId'])
token = resp['Token']
print "\nToken: %s"%(token)
print "\nIdentity ID: %s"%(resp['IdentityId'])
resp = client.get_credentials_for_identity(IdentityId=resp['IdentityId'])
secretKey = resp['Credentials']['SecretKey']
accessKey = resp['Credentials']['AccessKeyId']
print "\nSecretKey: %s"%(secretKey)
print "\nAccessKey ID: %s"%(accessKey)
print resp
conn = boto.connect_s3(aws_access_key_id=accessKey,aws_secret_access_key=secretKey,debug=0)
print "\nConnection: %s"%(conn)
for bucket in conn.get_all_buckets():
    print bucket.name

Error:

   Traceback (most recent call last):
  File "./test.py", line 32, in <module>
    for bucket in conn.get_all_buckets():
  File "/usr/local/lib/python2.7/dist-packages/boto/s3/connection.py", line 440, in get_all_buckets
    response.status, response.reason, body)
boto.exception.S3ResponseError: S3ResponseError: 403 Forbidden
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>InvalidAccessKeyId</Code><Message>The AWS Access Key Id you provided does not exist in our records.</Message><AWSAccessKeyId>ASIAILXMPZEMJAVZN7TQ</AWSAccessKeyId><RequestId>10631ACFF95610DD</RequestId><HostId>PGWDRBmhLjjv8Ast8v6kVHOG3xR8erJRV2ob3/2RmqHXwrg8HCZV578YsNLaoL24Hknr+nh033U=</HostId></Error>

This corresponding iOS code works fine:

AWSCognitoCredentialsProvider *credentialsProvider =
[AWSCognitoCredentialsProvider credentialsWithRegionType:awsCognitoRegionType
                                               accountId:awsAccountId
                                          identityPoolId:awsCognitoIdentityPool
                                           unauthRoleArn:unauthRoleArn
                                                  authRoleArn:nil];

AWSServiceConfiguration *configuration = [AWSServiceConfiguration configurationWithRegion:awsCognitoRegionType
                                                                      credentialsProvider:credentialsProvider];

....

AWSS3TransferManagerUploadRequest *uploadRequest = [AWSS3TransferManagerUploadRequest new];
uploadRequest.bucket = [ELEEnvironment currentEnvironment].userDataS3Bucket;
uploadRequest.key = key;
uploadRequest.body = uploadFileURL;
[[self uploadTask:uploadRequest] continueWithExecutor:[BFExecutor mainThreadExecutor]...

Thanks for any help!

Answer 1

This question is really invalid because the authentication was failing not on creating a session but when trying to list the buckets.

Uploading and downloading from a specific bucket works fine with the above code but not the listing of all buckets.

# Upload a new file
data = open('test.jpg', 'rb')
s3.Bucket('mybucket').put_object(Key='test.jpg', Body=data)

# S3 Object
obj = s3.Object(bucket_name='mybucket', key='test.jpg')
response = obj.get()
data = response['Body'].read()
print len(data)

Answer 2

PhilBot, I don't know why your original code sample connects to s3 using boto (as opposed to boto3). The code connects to cognito using boto3. As of now, boto3 is stable and there's probably not much reason to use boto anymore. (Maybe when you originally posted your question, boto3 was not as stable as it is today.)

When I tried using your code to connect to kinesis with boto3, it didn't work -- I had to pass response["Credentials"]["SessionToken"] as the aws_session_token to the client() function.