Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Web Worker consumes massive amount of memory

Tags:

I am trying to improve performance in my app which compares content of two large canvases.

The problem is that during comparison main thread of the browser is blocked what results unresponsive UI. Execution time of comparison function takes approx 10ms and comparison happens every 250ms or 500ms.

To solve that problem I came up with idea of use another thread by creating a web worker. It works pretty nice right now, but I realized that it consumes insane amount of memory (up to 600MB in FF and Chrome - in Edge it's up to 70MB, it never reached 100MB in single thread version on all browsers).

I thought that I left some references and Garbage Collector can't release memory. However, after some time (whole day of stripping my app to the pieces and trying to delete, null or undefined variables/data) I created fiddle below in which I'm sending 1MB ArrayBuffer (even though I passed it by transferring) but without any processing and as you can see it consumes massive amount of memory too.

Any ideas how I can solve that problem (any alternative solutions or any possible worker improvements) and where the problem is?

var sortFilterDataWorker = function () {
    onmessage = function image2compare(ev) {
      postMessage('hi');
    };
}.toString();

/* PREPARE WORKER AS STRING TO CREATE JS BLOB FILE */
sortFilterDataWorker = sortFilterDataWorker.slice(sortFilterDataWorker.indexOf('{') + 1, -1).trim();

var blob = new Blob([sortFilterDataWorker]) // create blob file with worker code
  , blobUrl = window.URL.createObjectURL(blob) // create pseudo url to blob file
  , compareWorker = new Worker(blobUrl) 
  ;

setInterval(function(){
  var oneMB = new ArrayBuffer(8388608);
  compareWorker.postMessage(oneMB, [oneMB]); // transpile ArrayBuffer
}, 250);

FIDDLE

EDIT:

I found out that if I terminate worker every n repeats and create new one releases memory faster. But it's still not a solution just a curio.

like image 629
LJ Wadowski Avatar asked Jan 25 '16 22:01

LJ Wadowski


People also ask

Is Web Worker secure?

The implementation of web workers ensures safe, conflict-free execution in two ways: A distinct, isolated global environment for the worker thread, separate from the browser environment. Pass-by-copy exchange of data between main and worker threads in the postMessage() call.

How many web workers web workers can run concurrently JavaScript?

A web worker is a JavaScript program running on a different thread, in parallel with main thread. The browser creates one thread per tab. The main thread can spawn an unlimited number of web workers, until the user's system resources are fully consumed.

Do web workers persist?

If only a single instance of the page is loaded then the connection count never changes when the page is reloaded or the link is clicked. So, in Chrome 20: Shared Web Workers do not persist across page reloads and link navigation clicks.


1 Answers

I found that if I run the garbage collector manually from developer tools -> Timeline it clears out all of the memory. Similarly, if I begin interacting with the Worker context from the console, calling functions seems to randomly trigger successful gc.

Based on this, I would say that there is not a hanging reference, but that receiving objects via a transfer may not force a gc check as new allocation requests would.

Transferring the object back with a response seems to workaround the problem:

postMessage('hi', [ev.data]);  // process usage stays around 50MB

As an alternative, making sure the Worker is non-trivial and will need to do normal allocations also seems to properly trigger gc, i.e:

  postMessage('hi');
  var twoMB = new ArrayBuffer(8388608); // usage cycles 70MB - ~220MB
like image 119
lossleader Avatar answered Sep 20 '22 21:09

lossleader