How to download a file from s3 using provided url?

Question

In my application I will get the url of s3 file like : https://s3.amazonaws.com/account-update/input.csv I have to download it and then process it. What I already done :

AmazonS3 s3 = new AmazonS3Client(credentials);
S3Object s3object = s3.getObject(new GetObjectRequest(
bucketName, key));

I am able to download the file by providing bucket name and key, but how can I download the file using the url(https://s3.amazonaws.com/account-update/input.csv) only?

Answer 1

You could download the file via a standard curl/wget, the same as you would download any other file off the Internet.

The important part, however, is to enable access to the object from Amazon S3. A few options:

• Make the object publicly-readable: This can be done via the console or CLI/API. However, anyone with that URL will be able to download it.
• Create an Amazon S3 Bucket Policy that grants read access for the desired file/directory/bucket. But, again, anyone with the URL will be able to access those objects.
• Keep the object private, but use a pre-signed URL that adds parameters to the URL to prove that you are permitted to download the object. This pre-signed URL is time-limited and can be generated with a few lines of code using current AWS credentials.

Answer 2

You may consider using the AWS SDK class AmazonS3URI as shown below:

URI fileToBeDownloaded = new URI(" https://s3.amazonaws.com/account-update/input.csv"); 

AmazonS3URI s3URI = new AmazonS3URI(fileToBeDownloaded);

S3Object s3Object = s3Client.getObject(s3URI.getBucket(), s3URI.getKey());

From here on, you should be able to utilise the s3Object obtained in a similar fashion to the s3Object shown in your code.

For more Java related AWS SDK examples on using this class check here