aws sdk Multipart Upload to s3 with node.js

Question

I am trying to upload large files to a s3 bucket using the node.js aws-sdk.

the V2 method upload integrally uploads the files in a multipart upload.

I want to use the new V3 aws-sdk. What is the way to upload large files in the new version? The method PutObjectCommand doesn't seem to be doing it.

I've seen there are methods such as CreateMultiPartUpload but I can't seem to find a full working example using them.

Thanks in advance.

Answer 1

As of 2021, I would suggest using the lib-storage package, which abstracts a lot of the implementation details.

Sample code:

import { Upload } from "@aws-sdk/lib-storage";
import { S3Client, S3 } from "@aws-sdk/client-s3";

const target = { Bucket, Key, Body };
try {
  const parallelUploads3 = new Upload({
    client: new S3({}) || new S3Client({}),
    tags: [...], // optional tags
    queueSize: 4, // optional concurrency configuration
    partSize: 5MB, // optional size of each part
    leavePartsOnError: false, // optional manually handle dropped parts
    params: target,
  });

  parallelUploads3.on("httpUploadProgress", (progress) => {
    console.log(progress);
  });

  await parallelUploads3.done();
} catch (e) {
  console.log(e);
}

Source: https://github.com/aws/aws-sdk-js-v3/blob/main/lib/lib-storage/README.md