Socket.io emit progress check

Question

I use Socket.io for uploading files. It works fine with the following schema:

1. Client: read file from input and encode it to Base64 and compress it
2. Client: Emit a Socket.io event "upload" including compressed file as a piece of data
3. Server: Listen to "upload" event, uncompress and decode file and save it

The problem occurs for large files: I cannot see a progress of data sent via client emit (like I do with XHR).

To solve these problems I have to track (check) upload progress of emit event. How can I do this?

---

I would like to listen to upload progress on CLIENT SIDE ONLY

---

Thanks to bnuhero, socketio-file-upload gives a possibility to listen to progress on server side and emit messages to client side when let's say every 5% or 20% is uploaded. So it means to send 20 or 5 messages respectively per file. I would like to listen to progress on client side.

---

Looks like there is no possibility to check the progress on client side with naked Socket.io. Io-stream solves that problem.

Answer 1

client:

var chunk_size = 100;
var compressed_data = 'some-long-string';
var reg = new RegExp('/.{1,'+chunk_size+'}/g');
var parts = compressed_data .match(reg);
var l = parts.length -1;

client.socket.emit('data', parts.pop());

client.socket.on('received', function () {
    client.socket.emit('data', [l - parts.lengt, parts.pop()]);
});

server:

sockets.on('connection', function (socket) {
  var parts = [];

  socket.on('data', function (data) {
     if (data[1] === undefined) {
         fs.writeFile(parts.join(''), callback...);
     } else {
         parts[data[0]] = data[1];
         socket.emit('received');
     }
  });


});

if this still doesnt work, you could add a negotiation between the server and client the server responsing with an "received" event and the client sending the next chunk on receiving the "recieved" event

that allows your server to throttle the speed of the client, this however should allready be working through the socket library you use

EDIT:

included feedback and message order

EDIT2:

I think i misunderstood your question

but to solve the issue as i understand it now, eg having a indication on when the buffer is flushed for each part it flushes there are 2 options, either have it still simulate that behavior, or accept the chunks the server accepts and use that as the size of each part.

The latter is the actual progress so il give an example of that (if the server accepts chunks of 1Mb and the file is 1Mb it will still go from 0 to 100 in one step tho.

https://github.com/nkzawa/socket.io-stream

the server example is just what is shown on that github page

the client:

var io = require('socket.io-client');
var ss = require('socket.io-stream');

var socket = io.connect('http://example.com/user');
var stream = ss.createStream();
var filename = 'profile.jpg';
var through = require('through');

var compressed_data = 'some-long-string';
var l = compressed_data .length;
var total_progress = 0;

//pass the stream trough throug, giving feedback for each chunk passed
var tr = through(function (chunk) {
    total_progress += chunk.toString().length;
    client.socket.emit('progress', l, total_progress);
    this.queue(chunk)
}, function () {
    client.socket.emit('progress', l, l);
})

//use the streaming version of socket.io
ss(socket).emit('profile-image', stream, {name: filename});

//get a stream for the compressed_data
//filter it trough tr for the progress indication
//and pass it to the socket stream
fs.createReadStream(compressed_data ).pipe(tr).pipe(stream);

streams have a backoff mechanism, the server accepts each chunk of the stream buffer if the server slows down the stream is read at a lower rate, giving you more progress inciations