Preemptive Proxy Authentication with HTTP Tunnel / HTTPS Connection in Apache HttpClient 4.3.1

Question

I am trying to send a HTTPS request through a proxy which needs preemptive authentication using Apache HttpClient 4.3.1.

My proxy blocks connections for some minutes from my IP when I am not directly authenticating myself in the first request.

I had no problems with normal HTTP requests, I just added the "Proxy-Authorization" header manually to the request.

But when attempting to load a HTTPS page, HttpClient seems to use a HTTP Tunnel so the first request is a "CONNECT" command and after that my actual request is sent. Using the request.setHeader(...) method does not affect the headers of the CONNECT request, resulting in a "HTTP/1.0 407 Proxy Authentication Required" response and the closing of my connection. After that, HttpClient again connects, this time adding the "Proxy-Authorization" header field with my credentials.

The connection succeeds (HTTP/1.0 200 Connection established) and my actual GET request is being executed. BUT when I run my program again after that, I'll get an IOException:

Information: I/O exception (java.net.SocketException) caught when processing request: Connection reset

In Wireshark I can see, that the proxy is not responding to my "CONNECT" requests (which do not contain credentials) anymore. So I tried several ways to get HttpClient to send credentials in the first CONNECT request: I adapted this example to use the proxy and created the AuthCache for the proxy, but it did not work. I also tried adding a HttpRequestInterceptor to my client:

static class PreemptiveAuth implements HttpRequestInterceptor {
    @Override
    public void process(final HttpRequest request, final HttpContext context) throws HttpException, IOException {
        request.setHeader("Proxy-Authorization", "Basic <base64credentials>");
    }
}

But this also does not affect the "CONNECT" requests. Here is the rest of my code:

public class ClientProxyAuthentication {

public static void main(String[] args) throws IOException, InterruptedException {
    HttpHost targetHost = new HttpHost("www.google.com", 443, "https");
    HttpHost proxy = new HttpHost("<proxy-ip>", 21265, "http");

    CredentialsProvider credsProvider = new BasicCredentialsProvider();
    credsProvider.setCredentials(
            new AuthScope("<proxy-ip>", 21265),
            new UsernamePasswordCredentials("username", "pass"));

    CloseableHttpClient httpclient = HttpClients.custom()
            .addInterceptorFirst(new PreemptiveAuth())
            .setProxy(proxy)
            .setProxyAuthenticationStrategy(new ProxyAuthenticationStrategy())
            .setDefaultCredentialsProvider(credsProvider).build();


    try {

        HttpGet httpget = new HttpGet("/");
        httpget.setHeader("Proxy-Authorization", "Basic <base64credentials>");

        System.out.println("executing request: " + httpget.getRequestLine());
        System.out.println("via proxy: " + proxy);
        System.out.println("to target: " + targetHost);

        CloseableHttpResponse response = httpclient.execute(targetHost, httpget);
        try {
            HttpEntity entity = response.getEntity();

            System.out.println("----------------------------------------");
            System.out.println(response.getStatusLine());
            if (entity != null) {
                System.out.println("Response content length: " + entity.getContentLength());
            }
            String html = EntityUtils.toString(entity, "UTF-8");
            System.out.println(html);
            EntityUtils.consume(entity);
        } finally {
            response.close();
        }
    } finally {
        httpclient.close();
    }
}

Answer 1

I suspect you have not correctly initialized auth cache. Please try this.

HttpHost proxy = new HttpHost("proxy", 8080);

BasicScheme proxyAuth = new BasicScheme();
// Make client believe the challenge came form a proxy
proxyAuth.processChallenge(new BasicHeader(AUTH.PROXY_AUTH, "BASIC realm=default"));
BasicAuthCache authCache = new BasicAuthCache();
authCache.put(proxy, proxyAuth);

CredentialsProvider credsProvider = new BasicCredentialsProvider();
credsProvider.setCredentials(
        new AuthScope(proxy),
        new UsernamePasswordCredentials("username", "password"));

HttpClientContext context = HttpClientContext.create();
context.setAuthCache(authCache);
context.setCredentialsProvider(credsProvider);

CloseableHttpClient httpclient = HttpClients.createDefault();
try {
    CloseableHttpResponse response = httpclient.execute(new HttpGet("/stuff"), context);
    try {
        // ...
    } finally {
        response.close();
    }
} finally {
    httpclient.close();
}