Apache HTTPClient Streaming HTTP POST Request?

Question

I'm trying to build a "full-duplex" HTTP streaming request using Apache HTTPClient.

In my first attempt, I tried using the following request code:

URL url=new URL(/* code goes here */);

HttpPost request=new HttpPost(url.toString());

request.addHeader("Connection", "close");

PipedOutputStream requestOutput=new PipedOutputStream();
PipedInputStream requestInput=new PipedInputStream(requestOutput, DEFAULT_PIPE_SIZE);
ContentType requestContentType=getContentType();
InputStreamEntity requestEntity=new InputStreamEntity(requestInput, -1, requestContentType);
request.setEntity(requestEntity);

HttpEntity responseEntity=null;
HttpResponse response=getHttpClient().execute(request); // <-- Hanging here
try {
    if(response.getStatusLine().getStatusCode() != 200)
        throw new IOException("Unexpected status code: "+response.getStatusLine().getStatusCode());

    responseEntity = response.getEntity();
}
finally {
    if(responseEntity == null)
        request.abort();
}

InputStream responseInput=responseEntity.getContent();
ContentType responseContentType;
if(responseEntity.getContentType() != null)
    responseContentType = ContentType.parse(responseEntity.getContentType().getValue());
else
    responseContentType = DEFAULT_CONTENT_TYPE;

Reader responseStream=decode(responseInput, responseContentType);
Writer requestStream=encode(requestOutput, getContentType());

The request hangs at the line indicated above. It seems that the code is trying to send the entire request before it gets the response. In retrospect, this makes sense. However, it's not what I was hoping for. :)

Instead, I was hoping to send the request headers with Transfer-Encoding: chunked, receive a response header of HTTP/1.1 200 OK with a Transfer-Encoding: chunked header of its own, and then I'd have a full-duplex streaming HTTP connection to work with.

Happily, my HTTPClient has another NIO-based asynchronous client with good usage examples (like this one). My questions are:

1. Is my interpretation of the synchronous HTTPClient behavior correct? Or is there something I can do to continue using the (simpler) synchronous HTTPClient in the manner I described?
2. Does the NIO-based client wait to send the whole request before seeking a response? Or will I be able to send the request incrementally and receive the response incrementally at the same time?

If HTTPClient will not support this modality, is there another HTTP client library that will? Or should I be planning to write a (minimal) HTTP client to support this modality?

Answer 1

Here is my view on skim reading the code:

1. I cannot completely agree with the fact that a non-200 response means failure. All 2XX responses are mostly valid. Check wiki for more details

2. For any TCP request, I would recommend to receive the entire response to confirm that it is valid. I say this because, a partial response may mostly be treated as bad response as most of the client implementations cannot make use of it. (Imagine a case where server is responding with 2MB of data and it goes down during this time)