How to create a child process from a string

Question

In the browser, we can create workers from a javascript string as follows:

var blob = new Blob([sourceString]);
var url = window.URL.createObjectURL(blob);
var worker = new Worker(url);

Is there any way to do this with node's child process? I have a single JavaScript file I and want to create workers that are coded dynamically.

The source string is a created dynamically at run time.

The closest answer I found was this one, but it requires a seperate file.

Answer 1

If I understood you right, I created a module which does just that yesterday.

It was not intended to create workers from strings but from actual functions even, because the actual function code must be passed thought message, they are stringified to be rebuilt (thought eval()) inside the worker.

And this is done thought the code:

var source = fn.toString();

...so, having that string prototype also has a .toString() method, passing the function as string must work too (and in fact works. I just tested it).

It may not be what you want: If you need to pass in and out messages from and to the worker, this module is not for you. But you can see the code and modify it to fit your needings.

On the other hand, if you only want to execute some function in background and get the result it is much simpler than dealing with worker plumbings because you can pass-in parameters to the function and get the result just as a simple function call.

Example:

// Reauires funwork (`npm install --save funwork`)

var funwork = require("funwork");
var workerfn = funwork(function_src_string); // or actual function.

It has the drawback that the function must be evaluated though eval() but, in your case, (having a string source) I think this is anyway a must.

EDIT: Here is a modified version of funwork to approach what you want as we discussed in comments:

var Worker = require('webworker-threads').Worker;
var Deasync = require('deasync');

function strWorker(fn){

    var source = fn.toString();

    return function() {

        var done = false;
        var args = Array.prototype.slice.call(arguments);
        var error;

        // Create worker://{{{
        var worker = new Worker(function(){
            var fn;
            var me = this;

            // Wait for function source and arguments:
            me.onmessage = function(event) {
                switch (event.data.oper) {
                    case "src":
                        // "Compile" function thougt source evaluation.
                        try {
                            eval ("fn = " + event.data.msg + ";");
                            postMessage(['ready']);
                        } catch (e) {
                            postMessage(['error', "Error trying to evaluate function source"]);
                        };
                        break;
                    case "args":
                        // Call the function with given arguments and reset the rest of worker stuff.
                        try {
                            // Reset worker (inside) event handler:
                            delete me.onmessage;

                            // Notify that worker is ready:
                            postMessage(["ok"]);

                            // Start function execution:
                            fn.apply(me, event.data.msg);

                        } catch (e) {
                            postMessage(['error', e]);
                        };
                        break;
                };
            };
        });//}}}

        // Event handling://{{{
        worker.onmessage = function(event) {
            switch (event.data[0]) {
                case 'error':
                    worker.postMessage({oper: "end"});
                    done = true;
                    error = event.data[1];
                    break;
                case 'ready':
                    worker.postMessage({oper: "args", msg: args});
                    break;
                case 'ok':
                    done = true;
                    break;
            };
        };//}}}

        // Send function source to worker:
        worker.postMessage({oper: "src", msg: source});

        // Wait (without blocking) until worker executed passed function:
        Deasync.loopWhile(function(){return !done;});

        if (error) throw error;

        // Reset worker (outside) event handler:
        delete worker.onmessage;

        return worker;
    };

};

module.exports = strWorker;

I kept the ability of passing arguments to the function because it is already implemented and you can simply don't use it if you doesn't need to pass anything.

The usage is the same with the only difference that the generated function returns a running worker instead of a function return value.

Used event handlers (inside and outside the worker ) are deleted prior to function (passed in as string) execution and worker returning, respectively, to avoid any side effect and the execution context ("this") of the passed-in function is also set to the actual worker "parent" function. .

Answer 2

If you want a single file js to spin up different processes creating a cluster might be a solution. Here is a pretty good tutorial: Tutorial

Basically node come with native cluster module

var cluster = require('cluster');

You can tell if the process is a master or a worker by cluster.isMaster. If the process is the master process you can spin up workers by doing cluster.fork()

if (cluster.isMaster) {
    for (var i = 0; i < numCPUs; i++) {
        cluster.fork();
    }
} else {
    http.createServer(function(req, res) {
        res.writeHead(200);
        res.end('process ' + process.pid + ' says hello!');
    }).listen(8000);
}

Hope this helps.

As for child process, you can child_process.fork(modulePath[, args][, options]) to run other modules and pass in arguments. The module can do different things according to the argument so it is dynamic... Seems you just want dynamic behavior based on the input and child_process can do it if you can make put the code in a different file. If you can only have one, try the cluster solution.