Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Getting file size from S3 bucket

I am trying to get the file size (content-length) using Amazon S3 JAVA sdk.

public Long getObjectSize(AmazonS3Client amazonS3Client, String bucket, String key)
        throws IOException {
    Long size = null;
    S3Object object = null;
    try {
        object = amazonS3Client.getObject(bucket, key);
        size = object.getObjectMetadata().getContentLength();

    } finally {
        if (object != null) {
            //object.close();
            1. This results in 50 calls (connection pool size) post that I start getting connection pool errors. 
            2. If this line is uncommented it takes hell lot of time to make calls.
        }
    }
    return size;
}

I followed this and this. But not sure what I am doing wrong here. Any help on this?

like image 995
Ashwani K Avatar asked Mar 08 '16 12:03

Ashwani K


People also ask

How can I get the size of an Amazon S3 bucket?

To find the size of a single S3 bucket, you can use the S3 console and select the bucket you wish to view. Under Metrics, there's a graph that shows the total number of bytes stored over time.

What is the size of object in S3?

Individual Amazon S3 objects can range in size from a minimum of 0 bytes to a maximum of 5 TB. The largest object that can be uploaded in a single PUT is 5 GB.

Does S3 storage scale automatically?

Amazon S3 automatically scales to high request rates. For example, your application can achieve at least 3,500 PUT/COPY/POST/DELETE or 5,500 GET/HEAD requests per second per partitioned prefix. There are no limits to the number of prefixes in a bucket.

How do I view an S3 bucket file?

In the Amazon S3 console, choose your S3 bucket, choose the file that you want to open or download, choose Actions, and then choose Open or Download. If you are downloading an object, specify where you want to save it.


2 Answers

I'm guessing what your actual question is asking, but I think you can reduce your code and eliminate the need to create an s3Object at all by doing something like:

public Long getObjectSize(AmazonS3Client amazonS3Client, String bucket, String key)
        throws IOException {
    return amazonS3Client.getObjectMetadata(bucket, key).getContentLength();
}

That should remove the need to call object.close() which you appear to be having issues with.

like image 123
Mark B Avatar answered Oct 21 '22 14:10

Mark B


For v2 of the Amazon S3 Java SDK, try something like this:

HeadObjectRequest headObjectRequest =
        HeadObjectRequest.builder()
          .bucket(bucket)
          .key(key)
          .build();
HeadObjectResponse headObjectResponse =
        s3Client.headObject(headObjectRequest);
Long contentLength = headObjectResponse.contentLength();
like image 31
kc2001 Avatar answered Oct 21 '22 13:10

kc2001