Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to close an AWS S3 read stream (AWSJavaScriptSDK)

I have an AWS S3 object and a read stream created on it like this:

const s3 = new AWS.S3();

const readStream = s3
  .getObject(params)
  .createReadStream()
  .on('error', err => {
    // do something
  });

Now when the stream is not read to the end (e.g. the streaming is aborted by client) after 120 sec the error event is triggered with: TimeoutError: Connection timed out after 120000ms

How can I close the stream (or the entire S3 object)?

I tried readStream.destroy() that is documented here, but it does not work.

like image 561
PeteMeier Avatar asked Jun 12 '18 11:06

PeteMeier


1 Answers

I was looking for a solution to a similar case and bumped into this thread.

There is an AWS Request abort method documented here which allows you to cancel the request without receiving all the data (it's a similar concept to node's http request).

Your code should look somewhat like this:

const s3 = new AWS.S3();

const request = s3.getObject(params);
const readStream = request.createReadStream()
  .on('error', err => {
    request.abort(); // and do something else also...
  });

It may be on error, but in my case - I'm fetching data and I want to stop streaming when I've reached a certain point (i.e. found specific data in the file and it's only a matter of checking if it exists - I don't need anything else).

The above will work well with both request and node-fetch modules as well.

like image 187
Michał Karpacki Avatar answered Nov 15 '22 03:11

Michał Karpacki