Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

S3 presigned download url immediately expired, why?

Tags:

amazon-s3

I have app that generates presigned url (using java sdk generatePresignedUrl method). Everything works on one environment (@ EU_central_1 server), but the same app published on other environment (client's EU_West_1) generates links that don't work, info from S3 when i try to download object right after creating an URL:

<Error>
   <Code>AccessDenied</Code>
   <Message>Request has expired</Message>
   <X-Amz-Expires>600</X-Amz-Expires>
   <Expires>2016-05-26T09:32:44Z</Expires>
   <ServerTime>2016-05-26T09:33:03Z</ServerTime>

As you can see, x-amz-expires was set to 600 seconds, but expires tag says object was expired immediately.

Is it a problem with GeneratePresignedUrlRequest.setExpiration that calculates incorrect expiration time ?

that's my code to set expiration time:

    Date expiration =  new Date();
    expiration.setTime(expiration.getTime() + 1000 * 600);
    GeneratePresignedUrlRequest generatePresignedUrlRequest = new GeneratePresignedUrlRequest(bucketName, key);
    generatePresignedUrlRequest.setMethod(HttpMethod.GET);
    generatePresignedUrlRequest.setExpiration(expiration);
    URL url = s3client.generatePresignedUrl(generatePresignedUrlRequest);

Looks like both servers return the same time. that's the response from two different EC2 servers connected to two different S3 servers, from the same area. One has a expire set to 4, second to 4000 (to be able to download resource right after creating a link).

Response from server working correctly:

<Error>
   <Code>AccessDenied</Code>
   <Message>Request has expired</Message>
   <X-Amz-Expires>4</X-Amz-Expires>
   <Expires>2016-05-31T09:54:04Z</Expires>
   <ServerTime>2016-05-31T11:00:17Z</ServerTime>

response from server with presigned URL problem:

<Error>
   <Code>AccessDenied</Code>
   <Message>Request has expired</Message>
   <X-Amz-Expires>4000</X-Amz-Expires>
   <Expires>2016-05-31T10:49:54Z</Expires>
   <ServerTime>2016-05-31T11:00:07Z</ServerTime>

both links created in the same time (with a few seconds difference for page refresh)

like image 644
razor Avatar asked May 26 '16 09:05

razor


People also ask

Why is my Presigned URL for an Amazon S3 bucket expiring before the expiration time that I specified?

If you created a presigned URL using a temporary token, then the URL expires when the token expires. The URL expires even if the URL was created with a later expiration time.

How long is S3 Presigned URL valid for?

Using the S3 console In the Amazon S3 console, the maximum expiration time for a presigned URL is 12 hours from the time of creation.

How many times can a Presigned URL be used?

It grants restricted access — only one of GET or PUT is allowed for a single URL. Only to a single object — each pre-signed URL corresponds to one object. With a time-constrained — the URL expires after a set timeout.

How does S3 Presigned URL work?

Pre-signed URLs are used to provide short-term access to a private object in your S3 bucket. They work by appending an AWS Access Key, expiration time, and Sigv4 signature as query parameters to the S3 object. There are two common use cases when you may want to use them: Simple, occasional sharing of private files.

What are presigned URLs in AWS S3?

Here’s were presigned URLs come in handy: AWS S3 provides an easy way to share S3 objects by creating signed (with owner credentials) links to access them. Amazon’s documentation explain this concept in a clear way: All objects by default are private. Only the object owner has permission to access these objects.

How long are S3 pre-signed URLs valid for?

It depends on how you generate the S3 pre-signed URL. Specifically, which signature version you use and what type of IAM credentials you use. The credentials that you can use to create a pre-signed URL include: IAM instance profile (temporary, rotated credentials): valid up to 6 hours

Why does Amazon S3 return this error message?

Amazon S3 returns this error message when you try to access a presigned URL past the specified expiration date of the URL. For example, in the following response, the expiration date of the presigned URL is June 28, 2018.

How to make a presigned url expire?

If you don't wanna expire your URL, you can very well use public objects. We can try some hack here , save image name and s3 bucket name into database instead of presigned url and while display create the "presigned" URLs with max expire time and send to browser. The maximum expiration time for presigned url is one week from time of creation.


2 Answers

Signature V4 (unlike V2) does not rely on the signature generation code to do the time math to figure out the expiration time.

Generating a V4 signature (as you are doing) requires that you know what time it is now, and include that value as X-Amz-Date. AWS then does the math on their side. "Hey, this guy says he signed it 11 minutes ago, and it's only good for 10 minutes... denied!"

Check the clock on the machine generating the signature.

like image 149
Michael - sqlbot Avatar answered Oct 17 '22 00:10

Michael - sqlbot


Please refer the below article to sync your time( between ec2 and s3) https://aws.amazon.com/blogs/aws/keeping-time-with-amazon-time-sync-service/

we need to use a service called chrony. its a versatile implementation on ntp and a bit more accurate and accomodate leap seconds.

Use below information to troubleshoot:

you can know the time in current linux machine using date command.

X-Amz-Date tells you when your url was signed in your ec2 or machine where the code is running. In the response,

<Expires>2016-05-31T10:49:54Z</Expires>
<ServerTime>2016-05-31T11:00:07Z</ServerTime>

Expires tell when the signature expired based on the X-Amz-Expires and ServerTime tells the time on the s3 server when the request was received.

like image 25
arvin_v_s Avatar answered Oct 17 '22 00:10

arvin_v_s