Is it possible somehow do multithreading in NodeJS?

Question

So i'm have an app with Socket.IO which purpose is to search some data on different sites. Something like crawler... The main problem is that the search process is too long and while it happens my app stucks... For example if one user starts to search second need to wait until first completed...

Each site which need to be searched is represented as a separate class so i do something like:

selected_sites.forEach(function(site_name) {
    var site = new sites[site_name];

    site.on('found', function(data) {
        socket.emit('found', data);
    });

    site.on('not_found', function() {
        socket.emit('not_found', 'Nothing found at ' + site.getSiteName());
    });

    site.search(socket_data.params);
});

Is it possible somehow to move the "class body | search progress" "somewhere else | in a new thread" so that event loop not be blocked while search in progress?

Answer 1

node.js does not allow you to run more threads of Javascript execution at the same time. A single node.js process only runs one Javascript thread of execution at a time. Because of asynchronous I/O, multiple Javascript operations may be "in flight" at any given time, but only one is actually running at any given time (while the others may be waiting for I/O operations to complete).

The usual way to solve a problem where you want some longer running and/or CPU intensive application to be run in the background while your server is free to handle incoming requests is to move the time consuming operation into it's own node.js process (often using the child process module) and then allow those two processes to share information as required, either via a database or via some interprocess communication like sockets.

If you have multiple CPU intensive operations, you can fire up multiple secondary processes or you can use the node.js clustering module in order to take maximum advantage of all CPUs in the host computer.

You should know that if most of your code is just networking or file I/O, then that can all be done with asynchronous operations and your node.js server will scale quite well to doing many different things in parallel. If you have CPU intensive operations (lots of parsing or calculations), then you will want to start up multiple processes in order to more effectively utilize multiple CPUs and let the system time slice the work for you.

Update in 2020: Nodejs now has threading. You can use Worker Threads. This would not be needed to parallelize I/O operations, but could be useful for paralellizing CPU-heavy operations and taking advantage of multiple CPU cores.