Tracking progress of file upload in multer nodejs

Question

How to track progress of a file getting uploaded to NodeJs server .I am using multer in server side to upload files. ?

Do i need to send some kind of information to the client , So that client gets the upload progress OR this is done internally & client can track progress on its on.

Below is the code i am using to upload file :

var multer = require('multer');
app.use(multer({dest:'./tmp',limits: {fileSize: 4*1024*1024}}).single('upload'));


router.post('/upload',function(req,res){
 console.log(req.file);
});

Answer 1

Multer is not very flexible when it comes to tracking the progress of a file upload. I've tried a lot of approaches, using the progress-stream library to pipe the request, making my own function to calculate the percentage and then stream the progress to socket.io, and other stuff. The approach with socket.io (streaming the percentage to a socket, back to the client) worked up to a certain point, but the main problem is that Multer does not return the control of execution until the file has been uploaded to the server, so if you have it set like a middleware function like:

this.router.post('/upload', upload.single('file'), function(req: Request, res: Response) {...}

there's no way you're going to get the progress or req.file (if you try adding a middleware before upload.single like this.router.post('/upload', dosomething, upload.single('file')...), because req.file will not exist at that point, and afterwards it will be 100% once you try to access req.file. That's just how middlewares work. If you need to track the progress, but without sending it back to the client (which is not the case if you need a progress bar), then you can do something like I did below. BUT! this will work because you will be calculating the request size, not the req.file size (the req.file only exists in the context of Multer) Not that it makes much of a difference, the req and req.file come in the same request, so if you use the code below, you will see the percentage and then the code for saving the file with Multer will be run. An example of tracking the progress on the server size, would be:

var uploadImages = multer({ dest: ...}).single('...');

//more code ...

this.router.post('/upload', (req: Request, res: Response) => {
    let progress = 0;
    let fileSize = req.headers['content-length'] ? parseInt(req.headers['content-length']) : 0;
    req.on('data', (chunk) => {
        progress += chunk.length;
        res.write((`${Math.floor((progress * 100) / fileSize)} `));
        if (progress === fileSize) {
            console.log('Finished', progress, fileSize)
        }
    });
})
//And calling the Multer function down here...

uploadImages(req,res...){}

Calling the middleware function to upload a file with Multer, instead of using it in the route declaration, is specified in Multer's documentation (where it says "Error handling"). "If you want to catch errors specifically from Multer, you can call the middleware function by yourself."

The value of req.file will only exist in the context where the data has reached Multer for processing and saving. So what I recommend, to anyone trying to come up with an approach where you listen for the progress on the server side, if you can, and need to do it on the front-end, you can use Axios, which has a very good hook to keep track of the progress, for promises.

Example of sending Form Data and keeping track of the progress:

saveFile(file,url): Promise<...>  {
    let formData = new FormData();
     formData.append('file', file);

     const config = {
        onUploadProgress: (progressEvent) => {
        var percentCompleted = Math.round((progressEvent.loaded * 100) / progressEvent.total)
            //do something with the percentCompleted
            //I used an observable to pass the data to a component and subscribed to it, to fill the progressbar
        }
       }

      return axios.post(url, formData, config)
}

Hope it works for your needs, and helps you to avoid the headaches.

Answer 2

Here's an answer by LinusU at the project github's page (he suggests using progress-stream):

Pipe req to that one and give it to multer.

    var p = progress()
    var upload = multer().single('file')

    req.pipe(p)
    p.headers = req.headers

    p.on('progress', _)
    upload(p, res, _)