Node.js Express - provide HTTP response before finish processing entire request

Question

I have a Node.js Express server where I send the response, and then attempt to do more processing with the same request instance.

If I write to res after headers are sent, I will get an error- but what happens if I use the req readable stream after I send back the response for the corresponding response?

In other words, how can I send an HTTP response with a Node.server before I finish processing the entire request?

In other other words, if I already have sent back a response, how can I "consume" the request after having already sent the response - is it really just a matter of doing anything besides send back a response?

Right now, there seem to be some weird errors related to using the request object (stream) that corresponds to the response that was already sent..

Let me give some examples, with code-

the following shows the last 3 Express middleware handlers in my server. As one interesting sidenote - once one middleware handler is invoked for a request, it can't be reinvoked (it appears so). So what happens is that the first handler below gets invoked when the response is sent. Then, the other path of execution (using process.nextTick or setImmediate) calls next(), and the second two handlers are invoked, which means I end up getting a 404 logged on my server.

app.use(function (req, res, next) {

    var r;

    var timeRequired = (Date.now() - req.request_start) + 'ms';
    console.log("Time required for request:", timeRequired);

    if (r = req.lectalTemp) {
        if (!req.lectalSent) {
            req.lectalSent = true;
            res.status(r.status).json({timeRequired: timeRequired, success: r.data});
        }
        else {
            console.log('Headers sent already, but here is the data that would have been sent:', r);
        }
    }
    else {
        next();
    }

});


// catch 404 and forward to error handler
app.use(function (req, res, next) {
    var err = new Error('404: Not Found - ' + req.method + ' ' + req.originalUrl);
    err.status = 404;
    next(err);
});

app.use(function (err, req, res, next) {

    var timeRequired = (Date.now() - req.request_start) + 'ms';

    if (app.get('env') === 'production') {
        res.status(err.status || 500).json({
            error: 'sorry the API experienced an error serving your priority request'
        });
    }
    else {
        console.error(colors.bgRed(err), '\n', err.stack);

        if (!res.headersSent && !req.lectalSent) {
            req.lectalSent = true;
            res.status(err.status || 500).json({
                error: {
                    timeRequired: timeRequired,
                    errMessage: err.message,
                    errStack: err.stack
                }
            });
        }

    }
});

Answer 1

I would have solved this with a queue. Post a message with the low critical data to a queue (eq. RabbitMq or similar) and create a worker that consume the messages in the queue asynchronously.