Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to download s3 object directly into memory in java

Tags:

amazon-s3

task

Is it possible to download S3object in Java directly into memory and get it removed when i'm done with the task?

like image 642
Manisha Avatar asked Nov 01 '12 14:11

Manisha


People also ask

How do I download files from AWS S3 to local system using Java?

S3Object fetchFile = s3. getObject(new GetObjectRequest(bucketName, fileName)); final BufferedInputStream i = new BufferedInputStream(fetchFile. getObjectContent()); InputStream objectData = fetchFile. getObjectContent(); Files.

How do I download S3 bucket items?

You can download an object from an S3 bucket in any of the following ways: Select the object and choose Download or choose Download as from the Actions menu if you want to download the object to a specific folder. If you want to download a specific version of the object, select the Show versions button.

How do I download from S3 bucket to local using command line?

Your answer You can use cp to copy the files from an s3 bucket to your local system. Use the following command: $ aws s3 cp s3://bucket/folder/file.txt . To know more about AWS S3 and its features in detail check this out!


2 Answers

Use the AWS SDK for Java and Apache Commons IO as such:

//import org.apache.commons.io.IOUtils  AmazonS3 s3  = new AmazonS3Client(credentials);  // anonymous credentials are possible if this isn't your bucket S3Object object = s3.getObject("bucket", "key");  byte[] byteArray = IOUtils.toByteArray(object.getObjectContent()); 

Not sure what you mean by "get it removed", but IOUtils will close the object's input stream when it's done converting it to a byte array. If you mean you want to delete the object from s3, that's as easy as:

s3.deleteObject("bucket", "key");  
like image 176
Zach Musgrave Avatar answered Sep 19 '22 18:09

Zach Musgrave


As of AWS JAVA SDK 2 you can you use ReponseTransformer to convert the response to different types. (See javadoc).

Below is the example for getting the object as bytes

GetObjectRequest request = GetObjectRequest.builder().bucket(bucket).key(key).build() ResponseBytes<GetObjectResponse> result = bytess3Client.getObject(request, ResponseTransformer.toBytes())  // to get the bytes result.asByteArray() 
like image 28
togise Avatar answered Sep 21 '22 18:09

togise