readable.on('end',...) is never fired

Question

I am trying to stream some audio to my server and then stream it to a service specified by the user, the user will be providing me with someHostName, which can sometimes not support that type of request.

My problem is that when it happens the clientRequest.on('end',..) is never fired, I think it's because it's being piped to someHostReq which gets messed up when someHostName is "wrong".

My question is:

Is there anyway that I can still have clientRequest.on('end',..) fired even when the stream clientRequest pipes to has something wrong with it?

If not: how do I detect that something wrong happened with someHostReq "immediately"? someHostReq.on('error') doesn't fire up except after some time.

code:

    someHostName = 'somexample.com'

    function checkIfPaused(request){//every 1 second check .isPaused
        console.log(request.isPaused()+'>>>>');
        setTimeout(function(){checkIfPaused(request)},1000);
    }

    router.post('/', function (clientRequest, clientResponse) {
        clientRequest.on('data', function (chunk) {
            console.log('pushing data');
        });

        clientRequest.on('end', function () {//when done streaming audio
            console.log('im at the end');
        }); //end clientRequest.on('end',)

        options = {
            hostname: someHostName, method: 'POST', headers: {'Transfer-Encoding': 'chunked'}
        };

        var someHostReq = http.request(options, function(res){
            var data = ''
            someHostReq.on('data',function(chunk){data+=chunk;});
            someHostReq.on('end',function(){
                console.log('someHostReq.end is called');
            });
        });
        clientRequest.pipe(someHostReq);
        checkIfPaused(clientRequest);
    });

output:

in the case of a correct hostname:

    pushing data
    .
    .
    pushing data
    false>>>
    pushing data
    .
    .
    pushing data
    pushing data
    false>>>
    pushing data
    .
    .
    pushing data
    console.log('im at the end');
    true>>>
    //continues to be true, that's fine

in the case of a wrong host name:

    pushing data
    .
    .
    pushing data
    false>>>>
    pushing data
    .
    .
    pushing data
    pushing data
    false>>>>
    pushing data
    .
    .
    pushing data
    true>>>>
    true>>>>
    true>>>>
    //it stays true and clientRequest.on('end') is never called
    //even tho the client is still streaming data, no more "pushing data" appears

if you think my question is a duplicate:

• it's not the same as this: node.js http.request event flow - where did my END event go? , the OP was just making a GET instead of a POST

• it's not the same as this: My http.createserver in node.js doesn't work? , the stream was in paused mode because the none of the following happened:

You can switch to flowing mode by doing any of the following:

Adding a 'data' event handler to listen for data.

Calling the resume() method to explicitly open the flow.

Calling the pipe() method to send the data to a Writable.

source: https://nodejs.org/api/stream.html#stream_class_stream_readable

• it's not the same as this: Node.js response from http request not calling 'end' event without including 'data' event , he just forgot to add the .on('data',..)

Answer 1

The behaviour in case of a wrong host name seems some problem with buffers, if the destination stream buffer is full (because someHost is not getting the sended chunks of data) the pipe will not continue to read the origin stream because pipe automatically manage the flow. As pipe is not reading the origin stream you never reach 'end' event.

Is there anyway that I can still have clientRequest.on('end',..) fired even when the stream clientRequest pipes to has something wrong with it?

The 'end' event will not fire unless the data is completely consumed. To get 'end' fired with a paused stream you need to call resume() (unpiping first from wrong hostname or you will fall in buffer stuck again) to set the steam into flowMode again or read() to the end.

But how to detect when I should do any of the above?

someHostReq.on('error') is the natural place but if it takes too long to fire up:

First try to set a low timeout request (less than someHostReq.on('error') takes to trigger, as seems too much time for you) request.setTimeout(timeout[, callback]) and check if it doesn't fail when correct hostname. If works, just use the callback or timeout event to detect when the server timeOut and use one of the techniques above to reach to the end.

If timeOut solution fails or doesn't fits your requirements you have to play with flags in clientRequest.on('data'), clientRequest.on('end') and/or clienteRequest.isPaused to guess when you are stuck by the buffer. When you think you are stuck just apply one of the techniques above to reach to the end of the stream. Luckily it takes less time to detect buffer stuck than wait for someHostReq.on('error') (maybe two request.isPaused() = true without reach 'data' event is enought to determine if you are stuck).

How do I detect that something wrong happened with someHostReq "immediately"? someHostReq.on('error') doesn't fire up except after some time.

Errors triggers when triggers. You can not "immediately" detect it. ¿Why not just send a prove beacon request to check support before piping streams? Some kind of:

"Cheking service specified by the user..." If OK -> Pipe user request stream to service OR FAIL -> Notify user about wrong service.