Howto properly dispose handles from child processes

Question

I am trying to build a platform to auto-deploy applications I build in NodeJS. Each application can be started and stopped from a control panel which handles these operations. The control panel does the following things when an application is launched:

• Requests a handle from a port-binder that is running as root. This port-binder is a forked child of the control panel.
• Now the control panel listens for requests on this port. Once a request is received it sends the handle of the client socket to the application that is listening on the 'message' event for new requests.
• Once the application is signaled the control panel releases all control over the request and socket.
• The application does what it needs to do and release and close the request and socket as well.

This setup works well, except when trying to bring down a server that is no longer required. Even though the server emits the 'close' event, it still accepts connection, which eventually time out because there is no requestListener bound anymore.

I'm assuming that there is a handle that is not being properly released and thus keeping the server awake.

Port binder:

var options = {
    http: require('http'),
    https: require('https')
};
process.on('message',function( data, handle ) {

    var port = data.port,
            type = data.type,
            server;

    server = options[type].createServer();
    server.once('error', function( err ) {
        process.send({success: false, port: port, error: err});
    });
    server.listen(port,function(){
        var handle = server._handle;

        process.send({success: true, port: port}, handle);

        server.close();
        handle.close();
        server._handle = null;
        handle = null;
        server.removeAllListeners();
    });
});

Control panel:

var cp = require('child_process'),
    app,
    types = {
        http: require('http'),
        https: require('https')
    },
    binder = cp.fork('./portbinder');

binder.on('message', function( data, handle ) {
    var server = types['http'].createServer(handler);

    server.once('close', function() {
        server._handle = null;
        handle = null;
    });
    server.listen( handle );
});

function handler( req, resp ) {
    app = app || cp.fork('./application'),
    socket = resp.socket,

    _handle = socket._handle,
    _req = {},
    _resp = {},
    _socket = {};

    // Copy transferable fields.
    for( i in req ) {
        if( typeof req[i] != 'function' && i != 'socket' && i != 'connection' && i != 'client' ) {
            _req[i] = req[i];
        }
    }

    for( i in resp ) {
        if( typeof resp[i] != 'function' && i != 'socket' && i != 'connection' && i != 'client' ) {
            _resp[i] = resp[i];
        }
    }

    for( i in socket ) {
        if( typeof socket[i] != 'function' &&
            i != '_handle' && i != '_httpMessage' &&
            i != '_idlePrev' && i != '_idleNext' && i != 'server' ) {
                _socket[i] = socket[i];
        }
    }

    // Send request to application.
    app.send({
        type: 'request',
        data: {
            req: _req,
            resp: _resp,
            socket: _socket
        }
    }, _handle );

    // Release the handle here without sending anything to the browser.
    socket._write = function() {};
    resp.end();
    socket.destroy();
    socket._handle = null;
    _handle = null;
};

Applications:

function reattach( target, properties ) {
    var i;

    for( i in properties ) {
        target[i] = properties[i];
    }

    return target;
};

function handler( req, resp ) {
    resp.end('hello world!');
};

process.on('message', function( param, handle ) {
    var _req = param.data.req,
        _resp = param.data.resp,
        _socket = param.data.socket,

    socket, resp, req;

    // Create a socket and reattach its properties.
    socket = new net.Socket({handle:handle});
    reattach( socket, _socket );

    // Create a request and reattach its properties.
    req = new http.IncomingMessage(socket);
    reattach( req, _req );

    // Create a response and reattach its properties.
    resp = new http.ServerResponse( req );
    resp.assignSocket(socket);
    reattach( resp, _resp );

    // Help closing down sockets after request is handled.
    resp.once('finish', function() {
        socket._write = function() {};
        socket.end();
        socket.destroy();
        socket._handle = null;
        handle = null;
    });

    handler( req, resp );
});

As you can see I'm trying to null and detach all the _handle properties I have, but the server handle does not stop listening on the port. When I make the controlpanel have an uncaught exception it crashes and the port binder does so as well. But the application node process does not quit and the port stays bound, so I'm geussing that I'm not properly releasing something there, but I'm unable to find it.

The entire setup does work, it's simply not closing down a server when I ask it to.

EDIT: I should probably note that I am using node.js v0.5.10

Answer 1

As it turns out it is a misinterpretation of the node.js documentation that led to this problem. The browser I was using sent a keep-alive header keeping connections open for some time to push new requests over the same connections.

When I did a server.close() I expected these connections to be closed because the docs say:

Stops the server from accepting new connections.

As it turns out, keep-alive connections are not closed, so the browser was still able to do requests as long as the connections was open giving me the idea that the server was still accepting connections.

The solution to this is reattaching a requestListener to your server after closing it. Then when you receive a connection, close the socket stream without sending anything. Now no new requests will be handled, and as soon as the connection times out it will close and the server will finally shutdown completely. And thus freeing up the port for other usages.

server.close();
// Drop any new request immediately, while server ditches old connections.
server.on('request', function( req, resp ) { req.socket.end(); });
server.once('close', function(){
    // Remove the listeners after the server has shutdown for real.
    server.removeAllListeners();
});

Putting this fix in place I could also clean up my code and remove the code to release the handle. node.js garbage collector now picks up the handle objects correctly and when the connection dies, so does the handle.

// All of this is no longer needed.
socket._write = function() {};
socket.end();
socket.destroy();
socket._handle = null;
handle = null;