Django collecstatic boto broken pipe on large file upload

Question

I am trying to upload the static files to my S3 bucket with collectstatic but i'm getting a broken pipe error with a 700k javascript file, this is the error

Copying '/Users/wedonia/work/asociados/server/asociados/apps/panel/static/panel/js/js.min.js'
    Traceback (most recent call last):
      File "manage.py", line 10, in <module>
        execute_from_command_line(sys.argv)
      File "/Users/wedonia/work/asociados/server/envs/asociados/lib/python2.7/site-packages/django/core/management/__init__.py", line 399, in execute_from_command_line
        utility.execute()
      File "/Users/wedonia/work/asociados/server/envs/asociados/lib/python2.7/site-packages/django/core/management/__init__.py", line 392, in execute
        self.fetch_command(subcommand).run_from_argv(self.argv)
      File "/Users/wedonia/work/asociados/server/envs/asociados/lib/python2.7/site-packages/django/core/management/base.py", line 242, in run_from_argv
        self.execute(*args, **options.__dict__)
      File "/Users/wedonia/work/asociados/server/envs/asociados/lib/python2.7/site-packages/django/core/management/base.py", line 285, in execute
        output = self.handle(*args, **options)
      File "/Users/wedonia/work/asociados/server/envs/asociados/lib/python2.7/site-packages/django/core/management/base.py", line 415, in handle
        return self.handle_noargs(**options)
      File "/Users/wedonia/work/asociados/server/envs/asociados/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 173, in handle_noargs
        collected = self.collect()
      File "/Users/wedonia/work/asociados/server/envs/asociados/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 112, in collect
        handler(path, prefixed_path, storage)
      File "/Users/wedonia/work/asociados/server/envs/asociados/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 317, in copy_file
        self.storage.save(prefixed_path, source_file)
      File "/Users/wedonia/work/asociados/server/envs/asociados/lib/python2.7/site-packages/django/core/files/storage.py", line 49, in save
        name = self._save(name, content)
      File "/Users/wedonia/work/asociados/server/envs/asociados/lib/python2.7/site-packages/storages/backends/s3boto.py", line 392, in _save
        self._save_content(key, content, headers=headers)
      File "/Users/wedonia/work/asociados/server/envs/asociados/lib/python2.7/site-packages/storages/backends/s3boto.py", line 403, in _save_content
        rewind=True, **kwargs)
      File "/Users/wedonia/work/asociados/server/envs/asociados/lib/python2.7/site-packages/boto/s3/key.py", line 1291, in set_contents_from_file
        chunked_transfer=chunked_transfer, size=size)
      File "/Users/wedonia/work/asociados/server/envs/asociados/lib/python2.7/site-packages/boto/s3/key.py", line 748, in send_file
        chunked_transfer=chunked_transfer, size=size)
      File "/Users/wedonia/work/asociados/server/envs/asociados/lib/python2.7/site-packages/boto/s3/key.py", line 949, in _send_file_internal
        query_args=query_args
      File "/Users/wedonia/work/asociados/server/envs/asociados/lib/python2.7/site-packages/boto/s3/connection.py", line 664, in make_request
        retry_handler=retry_handler
      File "/Users/wedonia/work/asociados/server/envs/asociados/lib/python2.7/site-packages/boto/connection.py", line 1068, in make_request
        retry_handler=retry_handler)
      File "/Users/wedonia/work/asociados/server/envs/asociados/lib/python2.7/site-packages/boto/connection.py", line 939, in _mexe
        request.body, request.headers)
      File "/Users/wedonia/work/asociados/server/envs/asociados/lib/python2.7/site-packages/boto/s3/key.py", line 842, in sender
        http_conn.send(chunk)
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 805, in send
        self.sock.sendall(data)
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 229, in sendall
        v = self.send(data[count:])
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 198, in send
        v = self._sslobj.write(data)
    socket.error: [Errno 32] Broken pipe

my settings are the following

DEFAULT_FILE_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
STATICFILES_STORAGE = 's3utils.StaticRootS3BotoStorage'  
AWS_S3_ACCESS_KEY_ID = ''
AWS_S3_SECRET_ACCESS_KEY = ''
AWS_STORAGE_BUCKET_NAME = 'asociados.static'

I tried change the internet connection and get the same result

I tried with a 200k file and got the same result

Someone have the same problem?

Answer 1

Set AWS_S3_HOST in settings.py to your correct region, e.g. s3-eu-west-1.amazonaws.com.

AWS_S3_HOST = "s3-eu-west-1.amazonaws.com"

See the latest Amazon list for complete reference.

This gist is also helpfull.