Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to check s3 object size using golang

I have implemented a function to download an object from AWS S3 bucket. This works fine. But I need to display a download progress bar. For this I need to know the size of the object beforehand according to here . Does anyone know how to get the object size?

Here is my code.

func DownloadFromS3Bucket(bucket, item, path string) {
    file, err := os.Create(filepath.Join(path, item))
    if err != nil {
        fmt.Printf("Error in downloading from file: %v \n", err)
        os.Exit(1)
    }

    defer file.Close()

    sess, _ := session.NewSession(&aws.Config{
        Region: aws.String(constants.AWS_REGION), Credentials: credentials.AnonymousCredentials},
    )

    // Create a downloader with the session and custom options
    downloader := s3manager.NewDownloader(sess, func(d *s3manager.Downloader) {
        d.PartSize = 64 * 1024 * 1024 // 64MB per part
        d.Concurrency = 6
    })

    numBytes, err := downloader.Download(file,
        &s3.GetObjectInput{
            Bucket: aws.String(bucket),
            Key:    aws.String(item),
        })
    if err != nil {
        fmt.Printf("Error in downloading from file: %v \n", err)
        os.Exit(1)
    }

    fmt.Println("Download completed", file.Name(), numBytes, "bytes")
}
like image 611
Madhuka Wickramapala Avatar asked Dec 24 '22 00:12

Madhuka Wickramapala


2 Answers

You can make use of the HeadObject, which contains the header Content-Length.

HeadObject API operation for Amazon Simple Storage Service.

The HEAD operation retrieves metadata from an object without returning the object itself. This operation is useful if you're only interested in an object's metadata. To use HEAD, you must have READ access to the object.

like image 184
hjpotter92 Avatar answered Jan 01 '23 09:01

hjpotter92


This should work:

headObj := s3.HeadObjectInput{
    Bucket: aws.String(bucket),
    Key: aws.String(key),
}
result, err := S3.HeadObject(&headObj)
if err != nil {
  // handle error
}
return aws.Int64Value(result.ContentLength)
like image 44
Andy Avatar answered Jan 01 '23 09:01

Andy