AWS S3 upload fails: RequestTimeTooSkewed

Question

I'm using

aws s3 sync ~/folder/ s3:// --delete

to upload (and sync) a large number of files to an S3 bucket. Some - but not all - of the files fail, throwing this error message:

upload failed: to s3://bucketname/folder/ A client error (RequestTimeTooSkewed) occurred when calling the UploadPart operation: The difference between the request time and the current time is too large

I know that the cause of this error is usually a local time that's out of sync with Internet time, but I'm running NTP (on my Ubuntu PC) and the date/time seem absolutely accurate - and this error has only been reported for about 15 out of the forty or so files I've uploaded so far. Some of the files are relatively large - up to about 70MB each - and my upload speeds aren't fantastic: could S3 possibly be comparing the initial and completion times and reporting their difference as an error? Thanks,

Answer 1

The time verification happens at the start of your upload to S3, so it won't be to do with files taking too long to upload.

Try comparing your system time with what S3 is reporting and see if there is any unnecessary time drift, just to make sure:

# Time from Amazon
$ curl http://s3.amazonaws.com -v

# Time on your local machine
$ date -u

(Time is returned in UTC)

Answer 2

I was running aws s3 cp inside docker container on a MacBook Pro and got this error. Restart the Docker for Mac fixed this issue.

Answer 3

Amazon S3 uses NTP for its system clocks, to sync with your clock. Run

sudo apt-get install ntp

then open /etc/ntp.conf and add at the bottom

server 0.amazon.pool.ntp.org iburst
server 1.amazon.pool.ntp.org iburst
server 2.amazon.pool.ntp.org iburst
server 3.amazon.pool.ntp.org iburst

Then run

sudo service ntp restart