What is the best approach for generating thumbnails and uploading to s3 bucket in node?

Question

I am developing a sample application where my back-end is on NodeJs and front-end uses React. This application is gonna deal with a bunch of photos, So user can upload photos and back-end will save it to s3 bucket. Additionally I need some transformations like generating thumbnails of different sizes most probably two thumbnails of 100 px * 100px and 200px * 200px.

With that being said, what would be best approach to do deal with thumbnail generation process ?

I also went through few stuff like cloudinary which seems be a good option for image transformation on the fly.

Now I am having a few questions on this related to the best practices.

1. Should I hit my app server(NodeJs) for image upload/transformation? The reason I am asking is, I tried out cloudinary where the image upload doesn't even hit app server.

2. Or it is fine to hit app server and generate the thumbnails there and upload it to s3.

3. Hit the app server and upload image to s3 bucket and trigger an lambda function to do all the transformations(What is the advantage ?)

Thanks for your patience for reading this questions. I am open to any suggestion, idea and concern.

Answer 1

It depends on your requirements, but I'll try and draw out the benefits and pitfalls of each.

First off, some things to consider:

• How many images are going to be uploaded?
• How often are those images and thumbnails going to be accessed?
• How quickly after upload do the thumbnails need to be accessible? Near to instantly as possible or is a few seconds delay acceptable?
• How large are these images?
• What's more important, running costs or development costs/time?

You'll need to understand all of these things to be able to compare effectively.

Your options aren't really three distinct options.

Option 1 is an independent architectural decision. Whether you push everything or anything back through your app server is up to you, there are sdks for AWS and Cloudinary for both the frontend or backend. But if you are putting everything in the frontend, be careful how you manage your keys. Well be careful either way, but it's typically the frontend code that people forget can be seen by everyone.

Options 2 and 3 relate to whether you do the additional work as part of the initial upload and under your control or separate concerns with more of an event based architecture.

If you do all the work on the app server, then make sure that it is of the right size to contend with the workload and that it can deal with retries if anything goes wrong. This is probably the most work for you and will obviously put the most load on your server. Upping the size of your server, will probably cost more than using lambdas unless your throughput is very constant, very high and you can spec them just right.

Having an automatic workflow on result of an image upload makes a bit of the complexity the responsibility of the cloud provider (in regards to retries at least). But you'll still need to get the lambda sizing right so it deal with everything in a timely manner.

There are plenty of code examples of both scenarios, but probably more for AWS lambda (caveat - based on 30 seconds of googling).

You haven't actually called it out as an option, but Cloudinary will do image manipulation on the fly (as you stated in your intro). This means that you don't actually need to create and save the thumbnails, just call the original image with the specified sizing in the url. Can you live with the delay? It's barely recognizable most of the time. Can you live with the cost? Whether it's money or using up your free credits.