Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

FileReader memory leak in Chrome

I have a webpage with file upload functionality. The upload is performed in 5MB chunks. I want to calculate hash for each chunk before sending it to the server. The chunks are represented by Blob objects. In order to calculate the hash I am reading such blob into an ArrayBuffer using a native FileReader. Here is the code:

var reader = new FileReader();

var getHash = function (blob, callback) {
    reader.onloadend = function (e) {
        var hash = util.hash(e.target.result);
        callback(hash);
    };
    reader.readAsArrayBuffer(blob);
}

var processChunk = function (chunk) {
    if (chunk) {
        getHash(chunk, function (hash) {
            util.sendToServer(chunk, hash, function() {
                // this callback is called when chunk upload is finished
                processChunk(chunks.shift());
            });
        });
    }
}

var chunks = file.splitIntoChunks(); // gets an array of blobs
processChunk(chunks.shift());

The problem: using the FileReader.readAsArrayBuffer seems to eat up a lot of memory which is not released. So far I tested with a 5GB file on following browsers:

  • Chrome 55.0.2883.87 m (64-bit): the memory goes up to 1-2GB quickly and oscillates around that. Sometimes it goes all the way up and browser tab crashes. It can use more memory than the size of read chunks. E.g. after reading 500MB of chunks the process already uses 700MB of memory.

  • Firefox 50.1.0: memory usage oscillates around 300-600MB

Code adjustments I have tried - all to no avail:

  • re-using the same FileReader instance for all chunks (as suggested in this question)
  • creating new FileReader for each chunk
  • adding timeout before starting new chunk
  • setting the FileReader and the ArrayBuffer to null after each read

The question: is there a way to fix the problem? Is this a bug in the FileReader implementations or am I doing something wrong?

EDIT: Here is a JSFiddle https://jsfiddle.net/andy250/pjt9udeu/

like image 412
andy250 Avatar asked Oct 17 '22 20:10

andy250


1 Answers

This is a bug in Chrome on Windows. It is reported here: https://bugs.chromium.org/p/chromium/issues/detail?id=674903

like image 187
andy250 Avatar answered Oct 20 '22 09:10

andy250