Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

golang s3 download to buffer using s3manager.downloader

Tags:

go

amazon-s3

I'm using the Amazon s3 SDK to download files like below:

file, err := os.Create("/tmp/download_file")
downloader := s3manager.NewDownloader(session.New(&aws.Config{
                                 Region: aws.String("us-west-2")}))
numBytes, err := downloader.Download(file,
    &s3.GetObjectInput{
        Bucket: aws.String(bucketName),
        Key:    aws.String(fileName),
    })

It downloads to a file.

How do I get the download content into a []byte slice (buffer) directly. I tried something like:

var tBuf bytes.Buffer
tBufIo := bufio.NewWriter(&tBuf)

instead of "file". But I get an error for io.WriterAt interface

cannot use *tBufIo (type bufio.Writer) as type io.WriterAt in argument to downloader.Download

like image 720
sriba Avatar asked Jan 14 '17 00:01

sriba


1 Answers

Found it from the link https://groups.google.com/forum/#!topic/Golang-Nuts/4z8rcWEZ8Os

buff := &aws.WriteAtBuffer{}
downloader := s3manager.NewDownloader(session.New(&aws.Config{
                                      Region: aws.String(S3_Region)}))
numBytes, err := downloader.Download(buff,....
data := buff.Bytes() // now data is my []byte array

Works and fits the need.

See also: https://docs.aws.amazon.com/sdk-for-go/api/aws/#WriteAtBuffer

like image 57
sriba Avatar answered Sep 22 '22 13:09

sriba