Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Getting s3 object metadata then creating stream

I'm downloading an object from s3 and creating a read stream object from it to process a video:

s3.getObject(params).createReadStream()

However, I need to get the metadata from it which is possible when i just get the object by accessing its 'metadata' property:

s3.getObject()

How would I either:

  1. Get the object via s3.getObject(), grab the metadata from its metadata property, and then turn it into a read stream?

    var stream = fs.createReadStream(response); isn't working - input must be a string

-- OR --

  1. Get the stream via s3.getObject().createReadStream(), and extract the metadata from the stream?

    To my knowledge metadata isn't passed within streams.


Tell me if my assumptions are wrong, but I am currently stuck with these two needs:

  • Getting the meta data
  • Making it a stream
like image 247
ian Avatar asked Feb 08 '23 03:02

ian


1 Answers

You can get the metadata via the request's httpHeaders event.

let fs = require('fs')
let aws = require('aws-sdk')
let s3 = new aws.S3()

let request = s3.getObject({
    Bucket: 'my-bucket',
    Key: 'my-key'
})

let stream

request.on('httpHeaders', (statusCode, httpHeaders) => {
    // object metadata is represented by any header in httpHeaders starting with 'x-amz-meta-'
    // you can use the stream object that this point
    stream.pipe(fs.createWriteStream('./somepath'))
    stream.on('end', () => { 
        console.log('were done')
    })
})

stream = request.createReadStream()

Alternatively you can also call s3.headObject to get the metadata without downloading the object and then download the object using s3.getObject

like image 125
cementblocks Avatar answered Feb 16 '23 03:02

cementblocks