botocore.exceptions.NoCredentialsError: Unable to locate credentials , Even after passing credentials manually

Question

Hi I am a newbie in creating flask application, i have created a small GUI to upload files to the S3 Bucket Here is the code snippet which is handling the same

s3 = boto3.client('s3', region_name="eu-west-1", 
    endpoint_url=S3_LOCATION, aws_access_key_id=S3_KEY, aws_secret_access_key=S3_SECRET)

myclient = boto3.resource('s3')
file = request.files['file[]']
filename=file.filename
data_files = request.files.getlist('file[]')
for data_file in data_files:
    file_contents = data_file.read()
    ts = time.gmtime()
    k=time.strftime("%Y-%m-%dT%H:%M:%S", ts)
    name=filename[0:-4]
    newfilename=(name+k+'.txt')
    myclient.Bucket(S3_BUCKET).put_object(Key=newfilename,Body=file_contents)
    message='File Uploaded Successfully'
    print('upload Successful')

the part is working fine when I am testing it from my local system, but upon uploading it to the EC2 Instance,the part

myclient.Bucket(S3_BUCKET).put_object(Key=newfilename,Body=file_contents)

is where it is throwing the error:

botocore.exceptions.NoCredentialsError: Unable to locate credentials

I have created a file config.py where I am storing the all the credentials and passing them at runtime. Not sure what is Causing the Error at EC2 instance, please help me with it

Answer 1

You may be confusing boto3 service resource and client.

#!!!! this instantiate an object call s3 to boto3.s3.client with credential  
s3 = boto3.client('s3', region_name="eu-west-1", 
    endpoint_url=S3_LOCATION, aws_access_key_id=S3_KEY, aws_secret_access_key=S3_SECRET)

#!!!! this instantiate an object call myclient to boto3.s3.resource that use
#     credential inside .aws folder, since no designated credential given.
myclient = boto3.resource('s3')

It seems you try to pass explicit credential to boto3.resource without using .aws/credential and .aws/default access key. If so, this is not the right way to do it. To explicitly pass the credential to boto3.resource, it is recommended to to use boto3.Session (that also works for boto3.client too). This also allow you to connect to different AWS services by using the initialise session than passing API key for different services inside your program.

import boto3
session = boto3.session(
      region_name = 'us-west-2', 
      aws_access_key_id=S3_KEY, 
      aws_secret_access_key=S3_SECRET)

# now instantiate the services
myclient = session.resource('s3')
# .... the rest of the code

Nevertheless, the better way is make use of .aws credential. Because it is a bad practice to hard code any access key/password inside the code. You can also use the profile name call if you need to access different API key in different region. e.g.

~/.aws/credential

[default]
aws_access_key_id = XYZABC12345
aws_secret_access_key = SECRET12345

[appsinfinity] 
aws_access_key_id = XYZABC12346
aws_secret_access_key = SECRET12346

~/.aws/config [default] region = us-west-1

[profile appsinfinity] 
region = us-west-2

And the code

import boto3
app_infinity_session = boto3.session(profile_name= 'appsinfinity')
....