Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to write a file from an ArrayBuffer in JS

I am trying to write a file uploader for Meteor framework. The principle is to split the fileon the client from an ArrayBuffer in small packets of 4096 bits that are sent to the server through a Meteor.method.

The simplified code below is the part of the client that sends a chunk to the server, it is repeated until offset reaches data.byteLength :

// data is an ArrayBuffer
var total = data.byteLength;
var offset = 0;

var upload = function() {
  var length = 4096; // chunk size

  // adjust the last chunk size
  if (offset + length > total) {
     length = total - offset;
  }

  // I am using Uint8Array to create the chunk
  // because it can be passed to the Meteor.method natively
  var chunk = new Uint8Array(data, offset, length);

  if (offset < total) {
     // Send the chunk to the server and tell it what file to append to
     Meteor.call('uploadFileData', fileId, chunk, function (err, length) {
        if (!err) {
          offset += length;
          upload();
        }
     }
  }
};
upload(); // start uploading

The simplified code below is the part on the server that receives the chunk and writes it to the file system :

var fs = Npm.require('fs');
var Future = Npm.require('fibers/future');

Meteor.methods({
  uploadFileData: function(fileId, chunk) {
    var fut = new Future();
    var path = '/uploads/' + fileId;

    // I tried that with no success
    chunk = String.fromCharCode.apply(null, chunk);

    // how to write the chunk that is an Uint8Array to the disk ?
    fs.appendFile(path, chunk, 'binary', function (err) {
      if (err) {
        fut.throw(err);
      } else {
        fut.return(chunk.length);
      }
    });
    return fut.wait();
  }
});

I failed to write a valid file to the disk, actually the file is saved but I cannot open it, when I see the content in a text editor, it is similar to the original file (a jpg for example) but some characters are different, I think that could be an encoding problem as the file size is not the same, but I don't know how to fix that...

like image 900
Karl.S Avatar asked Jul 23 '15 07:07

Karl.S


People also ask

How do I write an array to a file in node JS?

To write an array to file with Node. js, we can create a write stream. const fs = require('fs'); const file = fs. createWriteStream('array.

What is the use of ArrayBuffer in Javascript?

The ArrayBuffer object is used to represent a generic, fixed-length raw binary data buffer. It is an array of bytes, often referred to in other languages as a "byte array".

Is Uint8Array an ArrayBuffer?

For instance: Uint8Array – treats each byte in ArrayBuffer as a separate number, with possible values from 0 to 255 (a byte is 8-bit, so it can hold only that much). Such value is called a “8-bit unsigned integer”. Uint16Array – treats every 2 bytes as an integer, with possible values from 0 to 65535.

How do you store an array buffer?

There are no ArrayBuffers in the JSON format (only strings, numbers, booleans, null , objects and arrays) so if you want to save an ArrayBuffer in JSON then you'll have to represent it in one of those types (probably a string or an array of numbers).


2 Answers

Saving the file was as easy as creating a new Buffer with the Uint8Array object :

// chunk is the Uint8Array object
fs.appendFile(path, Buffer.from(chunk), function (err) {
    if (err) {
      fut.throw(err);
    } else {
      fut.return(chunk.length);
    }
});
like image 119
Karl.S Avatar answered Oct 08 '22 08:10

Karl.S


Building on Karl.S answer, this worked for me, outside of any framework:

fs.appendFileSync(outfile, Buffer.from(arrayBuffer));
like image 20
toms Avatar answered Oct 08 '22 07:10

toms