Resumable.js can't save file to server side directory

Question

I'm using Resumable.js with my Node.js-express application. I can get to the point where I know the file transfer was successful because I can console.log(req.body) and see this below...

{ resumableChunkNumber: '77',
  resumableChunkSize: '1048576',
  resumableCurrentChunkSize: '1064195',
  resumableTotalSize: '80755971',
  resumableType: '',
  resumableIdentifier: '80755971-humanfastq',
  resumableFilename: 'human.fastq',
  resumableRelativePath: 'human.fastq',
  resumableTotalChunks: '77' }

The documentation is rather vague about what to do after this point. I need to save this file to a directory on my server. Any help about how to accomplish this would be greatly appreciated. Thanks!

Answer 1

Not an expert but, I did give it a try and got an example working for you. You have not provided enough information in your question, so I am going to start from scratch.

I followed the official github repo and the samples provided and I had to make some tweaks in order to get the file chunks saved in a specific directory, then more tweaks to stitch those chunks back together and more tweak to delete the unnecessary chunks afterwards. Obviously all these tweaks are hints provided in the code, and elsewhere that I browsed. Appearntly flow.js is also using a similar mechanism (not entirely sure).

I basically needed to change the app.js in one of the samples in order to be able to upload files from browser and then save them locally.

1. Pass directory location to resumable where the file chunks should be added
2. Provide necessary conditions and commands so that when chunks are uploaded, you create a write stream and then use resumable to stitch the chunks together and save them as original file.
3. Finally delete all the chunks

Instructions to use the gist to get the express-app working (you can checkout the official github sample and make changes or you can follow below instructions):

1. Create a directory
2. Create the app.js, resumable.js and resumable-node.js from gist in the directory
3. copy/paste the package.json and do a npm install to get the modules (or manually npm install these express, path, fs, connect-multiparty)
4. create uploads directory in the main app directory and make sure it is writable
5. Copy/past the public directory from here that contains icons in the form of .png files and a style.css and index.html for uploading
6. Change the target value in the line 49 or 50 of index.html in the public directory you just copied to your servers address in my case http://localhost:3000/upload
7. You can tweak the chunk size, number of simultaneous chunk uploads, etc in the index.html but, I left them as default (in fact, I changed the chunks from 3 to 4)
8. Done all the above, then you are good to go.
9. Run the app i.e. nodemon app.js or node app.js
10. Navigate to your server address or http://localhost:3000/upload and you will see the index.html rendered as shown below:

The console log below (notice the end - chunks removed.)

And finally I have image2.jpg in the uploads directory.

TL;DR

You need the following tweaks to the app.js and index.html file to be able to upload and save uploaded file via Resumable.js.

index.html has been tweaked to change the target value to point it to the server url in my case http://localhost:3000/upload

var r = new Resumable({
  target:'http://localhost:3000/upload',
  chunkSize:1*1024*1024,
  simultaneousUploads:4,
  testChunks:false,
  throttleProgressCallbacks:1
});

app.js has been tweaked to send resumable a directory where to save the file (top most).

var resumable = require('./resumable-node.js')(__dirname + "/uploads");

I also tweaked app.js to change the content of app.post('/uploads',...) see gist.

// Handle uploads through Resumable.js
app.post('/upload', function(req, res){
  resumable.post(req, function(status, filename, original_filename, identifier){
    if (status === 'done') {
      var stream = fs.createWriteStream('./uploads/' + filename);

      //stich the chunks
      resumable.write(identifier, stream);
      stream.on('data', function(data){});
      stream.on('end', function(){});

      //delete chunks
      resumable.clean(identifier);
    }
    res.send(status, {
        // NOTE: Uncomment this funciton to enable cross-domain request.
        //'Access-Control-Allow-Origin': '*'
    });
  });
});

And finally last tweak to the app.js is at the last route handler, the following file

s.createReadStream("./resumable.js").pipe(res);

I moved the file resumable.js to the same directory as others, so I needed to tweak change it's location so that createReadStream find it.

Answer 2

I'm not using Resumable.js but something similar and here is how I do it in a Nodejs/express server. You can also take a look at more of the servers code here: https://github.com/MichaelLeeHobbs/roboMiner/tree/master/server/api/world however do keep in mind it is not complete and is experimental. The below code is only a bare example.

    import move from '../../libraries/fsMove.js';

    export function create(req, res) {
      var file = req.files.file;
      console.log(file.name);
      console.log(file.type);
      console.log(file.path);
      console.log(__dirname);
      move(file.path, __dirname + '/../../uploads/worlds/' + file.originalFilename, function(err){
        if (err) {
          console.log(err);
          handleError(res)(err);
          return;
        }
        World.createAsync(req.body)
          .then(responseWithResult(res, 201))
          .catch(handleError(res));
      });
    }

    // fsMove.js
    // http://stackoverflow.com/questions/8579055/how-i-move-files-on-node-js/29105404#29105404
    var fs = require('fs');

    module.exports = function move (oldPath, newPath, callback) {
      fs.rename(oldPath, newPath, function (err) {
        if (err) {
          if (err.code === 'EXDEV') {
            copy();
          } else {
            callback(err);
          }
          return;
        }
        callback();
      });

      function copy () {
        var readStream = fs.createReadStream(oldPath);
        var writeStream = fs.createWriteStream(newPath);

        readStream.on('error', callback);
        writeStream.on('error', callback);
        readStream.on('close', function () {

          fs.unlink(oldPath, callback);
        });

        readStream.pipe(writeStream);

      }
    };