Close Java HTTP Client

Question

Is there a way to close java.net.http.HttpClient to instantly release resources held by it?

Internally it holds a selector, a connection pool and an Executor (when default one is used). However it does not implement Closeable/AutoCloseable.

Answer 1

I had similar problem when I was redeploying a war file into Tomcat. War application had an HttpClient, that was running scheduled jobs issuing http requests and processing results.

I was seeing warnings from Tomcat when redeploying war file quite often on dev enrionment about hanging threads that may cause memory leaks. The stack trace was pointing to HttpClient threads. After several attempts I solved this problem in this way:

1. HttpClient is created only when necessary to execute the job. It is not created as a field of a class or serivec, only as a local variable inside the scheduled method.

2. HttpClient is created using builder and populated with a ThreadPool Executor, thus I keep the link to Executor and have control on it. ExecutorService executor = Executors.newSingleThreadExecutor(); HttpClient client = HttpClient.newBuilder().followRedirects(Redirect.ALWAYS).connectTimeout(Duration.ofSeconds(5)).executor(executor).build();

3. When job is done in try-catch block, finally section has these two lines that: close the thread pool explicitly and set null to the httpClient local variable: executor.shutdownNow(); client = null; System.gc();

Notes, have short connection timeout to limit the time of execution. keep number of threads small. I use threadPool of 1 thread.

After all these changes warnings about memory leaks disappeared from Tomcat logs.

Answer 2

As you've noticed, java.net.http.HttpClient does not implement Closeable or AutoCloseable. So I can think of only 2 options, but neither of them are really bulletproof or even good:

You could eliminate every strong reference to the HttpClient that your program is holding and request a garbage collection. However there's a real risk that something beyond your direct control is holding onto it or one of its components. Any remaining strong references would prevent the referenced object, and any objects it holds a strong reference to, from being garbage collected. Nevertheless, this is arguably the more idiomatic option than the alternative.

I also found another option.

final class HttpClientImpl extends HttpClient implements Trackable {
    ...
    // Called from the SelectorManager thread, just before exiting.
    // Clears the HTTP/1.1 and HTTP/2 cache, ensuring that the connections
    // that may be still lingering there are properly closed (and their
    // possibly still opened SocketChannel released).
    private void stop() {
        // Clears HTTP/1.1 cache and close its connections
        connections.stop();
        // Clears HTTP/2 cache and close its connections.
        client2.stop();
    }
    ...
}

I wouldn't feel comfortable using this unless I had no other choice. Your reference is probably of type HttpClient, so you'd need to cast it to HttpClientImpl. It's bad to rely on the concrete implementation, which could change in future releases, rather than the HttpClient interface. The method is also private. There are ways around this but it's messy.

Answer 3

In Java 11, each HttpClient spawn a daemon thread called selmgr which supposed to take care of in fly requests. This thread will be closed when there is no reference to the HttpClient in the code. However, in my experience, it is not reliable. Especially, when you use asynchronous methods with future timeouts.

Here is a piece of code I wrote using reflection to reliably close the HttpClient

static void shutDownHttpClient(HttpClient httpClient)
{
    ThreadPoolExecutor threadPoolExecutor = (ThreadPoolExecutor) httpClient.executor().get();
    threadPoolExecutor.shutdown();
    try {
        Field implField = httpClient.getClass().getDeclaredField("impl");
        implField.setAccessible(true);
        Object implObj = implField.get(httpClient);
        Field selmgrField = implObj.getClass().getDeclaredField("selmgr");
        selmgrField.setAccessible(true);
        Object selmgrObj = selmgrField.get(implObj);
        Method shutDownMethod = selmgrObj.getClass().getDeclaredMethod("shutdown");
        shutDownMethod.setAccessible(true);
        shutDownMethod.invoke(selmgrObj);
    }
    catch (Exception e) {
        System.out.println("exception " + e.getMessage());
        e.printStackTrace();
    }

}

As you can see, this is implementation-dependent and may not works with future Java versions. It is tested with Java 11 and Java 12.

Also, you need to add --add-opens java.net.http/jdk.internal.net.http=ALL-UNNAMED to your java command.

Answer 4

Obviously HttpClient is designed to be self managed. So it responsible to maintain connection pool, cache ttl by itself.

In HttpClientCode We could find the following code:

if (!owner.isReferenced()) {
                                Log.logTrace("{0}: {1}",
                                        getName(),
                                        "HttpClient no longer referenced. Exiting...");
                                return;
                            }

this is a graceful way to exit from SelectorManager loop and clean all resources.

 @Override
 public void run() {
            ...

            try {

                while (!Thread.currentThread().isInterrupted()) {

                    ...

                    if (!owner.isReferenced()) {
                        Log.logTrace("{0}: {1}",
                                getName(),
                                "HttpClient no longer referenced. Exiting...");
                        return;
                    }

                    ...
                }
            } catch (Throwable e) {
                ...
            } finally {
                ...
                shutdown();
            }
        }




    final boolean isReferenced() {
            HttpClient facade = facade();
            return facade != null || referenceCount() > 0;
        }

So when your HttpClient object will not be referenced, then it will clean all resources.

UPD: also you should tune your requests by passing timeouts