[Django][AWS S3] botocore.exceptions.clienterror an error occurred (accessdenied) when calling the PutObject operation

Question

I am trying to connect Django project to AWS S3.

settings.py contains below:

AWS_ACCESS_KEY_ID = #ID
AWS_SECRET_ACCESS_KEY = #Key
AWS_STORAGE_BUCKET_NAME = #Bucket
AWS_S3_CUSTOM_DOMAIN = '%s.s3.amazonaws.com' % AWS_STORAGE_BUCKET_NAME
AWS_S3_OBJECT_PARAMETERS = {
    'CacheControl': 'max-age=86400',
}
AWS_LOCATION = 'static'

STATICFILES_DIRS = [
    os.path.join(BASE_DIR, 'backend/static'),
]
STATIC_URL = 'https://%s/%s/' % (AWS_S3_CUSTOM_DOMAIN, AWS_LOCATION)
STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'

IAM user is created with AmazonS3FullAccess. But when I enter:

python manage.py collectstatic

an error occurs:

You have requested to collect static files at the destination location as specified in your settings.

This will overwrite existing files! Are you sure you want to do this?

Type 'yes' to continue, or 'no' to cancel: yes Traceback (most recent call last): File "manage.py", line 22, in execute_from_command_line(sys.argv) File "/home/seokchan/server/mdocker/lib/python3.5/site-packages/django/core/management/init.py", line 381, in execute_from_command_line utility.execute() File "/home/seokchan/server/mdocker/lib/python3.5/site-packages/django/core/management/init.py", line 375, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/home/seokchan/server/mdocker/lib/python3.5/site-packages/django/core/management/base.py", line 316, in run_from_argv self.execute(*args, **cmd_options) File "/home/seokchan/server/mdocker/lib/python3.5/site-packages/django/core/management/base.py", line 353, in execute output = self.handle(*args, **options) File "/home/seokchan/server/mdocker/lib/python3.5/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 188, in handle collected = self.collect() File "/home/seokchan/server/mdocker/lib/python3.5/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 114, in collect handler(path, prefixed_path, storage) File "/home/seokchan/server/mdocker/lib/python3.5/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 353, in copy_file self.storage.save(prefixed_path, source_file) File "/home/seokchan/server/mdocker/lib/python3.5/site-packages/django/core/files/storage.py", line 49, in save return self._save(name, content) File "/home/seokchan/server/mdocker/lib/python3.5/site-packages/storages/backends/s3boto3.py", line 506, in _save self._save_content(obj, content, parameters=parameters) File "/home/seokchan/server/mdocker/lib/python3.5/site-packages/storages/backends/s3boto3.py", line 521, in _save_content obj.upload_fileobj(content, ExtraArgs=put_parameters) File "/home/seokchan/server/mdocker/lib/python3.5/site-packages/boto3/s3/inject.py", line 621, in object_upload_fileobj ExtraArgs=ExtraArgs, Callback=Callback, Config=Config) File "/home/seokchan/server/mdocker/lib/python3.5/site-packages/boto3/s3/inject.py", line 539, in upload_fileobj return future.result() File "/home/seokchan/server/mdocker/lib/python3.5/site-packages/s3transfer/futures.py", line 106, in result return self._coordinator.result() File "/home/seokchan/server/mdocker/lib/python3.5/site-packages/s3transfer/futures.py", line 265, in result raise self._exception File "/home/seokchan/server/mdocker/lib/python3.5/site-packages/s3transfer/tasks.py", line 126, in call return self._execute_main(kwargs) File "/home/seokchan/server/mdocker/lib/python3.5/site-packages/s3transfer/tasks.py", line 150, in _execute_main return_value = self._main(**kwargs) File "/home/seokchan/server/mdocker/lib/python3.5/site-packages/s3transfer/upload.py", line 692, in _main client.put_object(Bucket=bucket, Key=key, Body=body, **extra_args) File "/home/seokchan/server/mdocker/lib/python3.5/site-packages/botocore/client.py", line 357, in _api_call return self._make_api_call(operation_name, kwargs) File "/home/seokchan/server/mdocker/lib/python3.5/site-packages/botocore/client.py", line 661, in _make_api_call raise error_class(parsed_response, operation_name) botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the PutObject operation: Access Denied

So, I edited bucket policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Allow All",
            "Effect": "Allow",
            "Principal": "*",
            "Action": [
                "s3:GetObject",
                "s3:PutObject",
                "s3:PutObjectAcl"
            ],
            "Resource": "arn:aws:s3:::<bucket name>/*"
        }
    ]
}

but the error still occurred. How can I resolve this error?

A tutorial that I am following doesn't show any error at this step.(https://simpleisbetterthancomplex.com/tutorial/2017/08/01/how-to-setup-amazon-s3-in-a-django-project.html)

Answer 1

It was AWS S3 access problem.

In S3 bucket console, I edited bucket's public access as public.

NB : Only do this if your intention is to make the file publicly available for example of you're using it to serve files for your website, like images, css etc things that everyone needs to have access to.

Answer 2

Setting AWS_DEFAULT_ACL = None worked for me. It looks like boto requests public-read ACL by default so unless you have made your bucket public it won't work.

Answer 3

This worked for me:

In my S3 bucket -> Permissions Tab -> click Block public access -> Edit -> untick Block all public access -> Save

AND

In my AWS IAM settings -> Users Tab (under Access Management) -> <my-user> -> Add Permissions -> add AmazonS3FullAccess

This granted the user (identified by AWS id and AWS secret) access to control my s3 buckets