Is JDK ClassLoader.getResourceAsStream broken? (unclosed resources)

Question

I will try to prove that ClassLoader.getResourceAsStream() is opening two InputStreams, closing none of it and returning only one to client. Is my logic correct? JDK sources are picked from jdk1.8.0_25

I've get into unclosed resources problem using Spring ClassPathResource in interval (original question), that is using ClassLoader.getResourceAsStream to get InputStream to a properties file.

After investigation, I found that classLoader.getResourceAsStream is getting an URL by URL url = getResource(name); and then it is opening that stream, but URL url = getResource(name) already opens that stream. JDK source of ClassLoader:

    public InputStream getResourceAsStream(String name) {
        URL url = getResource(name); /* SILENTLY OPENS AND DON'T CLOSES STREAM */
        try {
            return url != null ? url.openStream() : null; /* SECOND OPEN !!! */
        } catch (IOException e) {
            return null;
        }
    }

If we will close() the InputStream provided that way, we will close only the stream opened by url.openStream(). JDK source:

    public final InputStream openStream() throws java.io.IOException {
        return openConnection().getInputStream();
    }

I'm supposing that, the problem is, the JDK opens a stream silently in URL url = getResource(name) only to get URL object that is used further to create **second (returned to client) stream**. Look at this method sources:

    public URL getResource(String name) {
        URL url;
        if (parent != null) {
            url = parent.getResource(name);
        } else {
            url = getBootstrapResource(name); <---- we end up calling that method
        }
        if (url == null) {
            url = findResource(name);
        }
        return url;
    }

And now, in getBootstrapResource(name) the moment when we convert Resource to URL forgetting about opened stream in Resource!:

private static URL getBootstrapResource(String name) {
    URLClassPath ucp = getBootstrapClassPath();
    Resource res = ucp.getResource(name); <---- OPENING STREAM [see further]
    return res != null ? res.getURL() : null; <--- LOSING close() CAPABILITY
}

Why ucp.getResource(name); is opening resource? Let's look into that method: this.getResource(var1, true);, which delegates to:

public Resource getResource(String var1, boolean var2) {
    if(DEBUG) {
        System.err.println("URLClassPath.getResource(\"" + var1 + "\")");
    }

    URLClassPath.Loader var3;
    for(int var4 = 0; (var3 = this.getLoader(var4)) != null; ++var4) {
        Resource var5 = var3.getResource(var1, var2); <-------- OPENING STREAM
        if(var5 != null) {
            return var5;
        }
    }

    return null;
}

Why Resource var5 = var3.getResource(var1, var2); is opening stream? Look further:

Resource getResource(final String var1, boolean var2) {
        final URL var3;
        try {
            var3 = new URL(this.base, ParseUtil.encodePath(var1, false));
        } catch (MalformedURLException var7) {
            throw new IllegalArgumentException("name");
        }

        final URLConnection var4;
        try {
            if(var2) {
                URLClassPath.check(var3);
            }

            var4 = var3.openConnection(); <------------ OPENING STREAM
            InputStream var5 = var4.getInputStream();
            if(var4 instanceof JarURLConnection) {
                JarURLConnection var6 = (JarURLConnection)var4;
                this.jarfile = URLClassPath.JarLoader.checkJar(var6.getJarFile());
            }
        } catch (Exception var8) {
            return null;
        }

        return new Resource() {
            public String getName() {
                return var1;
            }

            public URL getURL() {
                return var3;
            }

            public URL getCodeSourceURL() {
                return Loader.this.base;
            }

            public InputStream getInputStream() throws IOException {
                return var4.getInputStream();
            }

            public int getContentLength() throws IOException {
                return var4.getContentLength();
            }
        };
    }

We can see openConnection() and getInputStream(), which are not closed, and falling back thrgough all the calls returning Resource we are finally using only the getURL() method wrapped in Resource without closing it's InputStream only to use that URL object to open jet another InputStream and return it to a client (which client can close of coruse, but we end with first stream unclosed).

So, is ClassLaoder.getResourceAsStream broken with leaking resources?

Practical side: I'm using getResourceAsStream in try-with-resources block, and still have unclosed resources problems in production with filename loaded such way every 30-seconds. More, all that resources are closed on garbage collection, which is consistent with file stream close() in finalize() method.

Answer 1

I made a simple test program to verify the actual behavior:

System.out.println(System.getProperty("java.version"));
URL testURL = new URL("test", null, 0, "/", new URLStreamHandler() {
    protected URLConnection openConnection(URL u) throws IOException {
        System.out.println("creating connection to "+u);
        return new URLConnection(u) {
            InputStream is;
            public void connect(){}
            @Override
            public InputStream getInputStream() throws IOException {
                System.out.println("getInputStream() for "+u);
                if(is==null) is=new InputStream() {
                    boolean open=true;
                    @Override
                    public void close() throws IOException {
                        if(!open) return;
                        System.out.println("One InputStream for "+u+" closed");
                        open=false;
                    }
                    public int read() { return -1; }
                };
                else System.out.println("COULD be shared");
                return is;
            }
        };
    }
});
System.out.println("\n  trying new ClassLoader");
try(URLClassLoader newlClassLoader=new URLClassLoader(new URL[]{ testURL });
    InputStream is=newlClassLoader.getResourceAsStream("foo")) {}

System.out.println("\n  trying System ClassLoader");
try {
    Method m=URLClassLoader.class.getDeclaredMethod("addURL", URL.class);
    m.setAccessible(true);
    m.invoke(ClassLoader.getSystemClassLoader(), testURL);
} catch(Exception ex) { ex.printStackTrace(); }
try(InputStream is=ClassLoader.getSystemResourceAsStream("foo")) {}

System.out.println("\n  trying bootstrap ClassLoader");
try {
    Method m=ClassLoader.class.getDeclaredMethod("getBootstrapClassPath");
    m.setAccessible(true);
    Object bootstrap = m.invoke(null);
    m=bootstrap.getClass().getDeclaredMethod("addURL", URL.class);
    m.setAccessible(true);
    m.invoke(bootstrap, testURL);
} catch(Exception ex) { ex.printStackTrace(); }

try(InputStream is=ClassLoader.getSystemClassLoader().getResourceAsStream("foo")) {}

on my machine using (tested with 1.8.0_05, 1.8.0_20 and 1.8.0_40) it printed

  trying new ClassLoader
creating connection to test:/foo
getInputStream() for test:/foo
One InputStream for test:/foo closed
creating connection to test:/foo
getInputStream() for test:/foo
One InputStream for test:/foo closed

  trying System ClassLoader
creating connection to test:/foo
getInputStream() for test:/foo
One InputStream for test:/foo closed
creating connection to test:/foo
getInputStream() for test:/foo
One InputStream for test:/foo closed

  trying bootstrap ClassLoader
creating connection to test:/foo
getInputStream() for test:/foo
creating connection to test:/foo
getInputStream() for test:/foo
One InputStream for test:/foo closed

So from this test, I can conclude that the resources are indeed opened twice but also correctly closed for all resources accessed via user class path and additional ClassLoaders, so there’s no resource leak in these cases.

Your code analysis regarding the bootstrap resource behavior is correct, there is a resource leak but usually this doesn’t occur for resources required by your application as these should be accessible via user class path. ClassLoaders try their parents first but your resource shouldn’t be found in the bootstrap class path, hence that attempt should return null and not open any resource.

So it’s crucial to ensure that application specific resources are not accessible via the JRE’s bootstrap class path, e.g. don’t manipulate the bootstrap class path and don’t put resources into the JRE’s extension directories. This applies also to the test code above, if you change the order of the tests, i.e. patch the bootstrap class path first, all tests will show a leak as all lookups try their parent first, ending at the bootstrap loader.