How can I disable default request headers from apache httpclient 4?

Question

I am using apache common httpclient 4.3.3 to make http 1.0 request. Here is how I make the request

HttpClient client = HttpClientBuilder.create().build();
HttpPost post = new HttpPost(url);
post.setProtocolVersion(new ProtocolVersion("HTTP", 1, 0));

 // trying to remove default headers but it doesn't work
post.removeHeaders("User-Agent");
post.removeHeaders("Accept-Encoding");
post.removeHeaders("Connection");

post.setEntity(new ByteArrayEntity(ba) );

HttpResponse response = client.execute(post);

However, i can see that there are other headers automatically added to my request to the server like

Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.3.3 (java 1.5)
Accept-Encoding: gzip,deflate

How can I tell httpclient not to include any other headers? I tried to removed those headers with post.removeHeaders(xxxx) but it doesn't work. Can you show me how?

Thanks,

Answer 1

If you call HttpClientBuilder.create(), you will have a httpClientBuilder. And httpClientBuilder has a lot config for default headers and this will be used to make intercepters( ex: RequestAcceptEncoding ).

For example, RequestAcceptEncoding, which implements HttpRequestInterceptor, makes Accept-Encoding: gzip,deflate header when HttpProcessor.process() is invoked. And httpProcessor.process() will be invoked just before invoking final CloseableHttpResponse response = this.requestExecutor.execute(route, request, context, execAware);

You can see this code at org.apache.http.impl.execchain.ProtocolExec of httpclient-4.3.6 line 193.

If you want to remove Accept-Encoding: gzip,deflate, call HttpClientBuilder.disableContentCompression() like below.

HttpClient client = HttpClientBuilder.create().disableContentCompression().build();

In short, HttpClientBuilder has a lot of flags to disable/enable HttpRequestInterceptor. If you disable/enable those HttpRequestInterceptor, you can exclude/include default headers.

Sorry for my poor English, and hope you get what I mean.

Answer 2

CloseableHttpClient hc = HttpClients.custom()
        .setHttpProcessor(HttpProcessorBuilder.create().build())
        .build();

The code snippet above demonstrates how to create an HttpClient instance with an empty (no-op) protocol processor, which guarantees no request headers will ever be added to outgoing messages executed by such client

Answer 3

You want to do your 'cleanup' at the end, after HttpClient is done modifying the request. You can do this by calling addInterceptorLast on HttpClientBuilder as below.

HttpClient client = HttpClientBuilder.create().addInterceptorLast( 
                        new HttpRequestInterceptor() {
                            public void process(HttpRequest request, HttpContext context){
                                request.removeHeaders("Host");
                                request.removeHeaders("Connection");
                            }
                        }   
                ).build();

I create an anonymous class implementing HttpRequestInterceptor. Take whatever header modifications you need done before the request is sent, and put them in the process method.