java.nio.file.Files.delete(Path path) - occasional failure to recursively delete directory using SimpleFileVisitor

Question

Trying to troubleshoot an occasional java.nio.file.DirectoryNotEmptyException in a recursive delete method taken from Delete directories recursively in Java

Code (credit to @TrevorRobinson) :

static void removeRecursive(Path path) throws IOException {
    Files.walkFileTree(path, new SimpleFileVisitor<Path>() {

        final Logger logger = LoggerFactory.getLogger(this.getClass());
        @Override
        public FileVisitResult visitFile(Path file,
                BasicFileAttributes attrs) throws IOException {
            logger.warn("Deleting " + file.getFileName());
            Files.delete(file);
            logger.warn("DELETED " + file.getFileName());
            return FileVisitResult.CONTINUE;
        }

        @Override
        public FileVisitResult visitFileFailed(Path file, IOException exc) {
            // try to delete the file anyway, even if its attributes could
            // not be read, since delete-only access is theoretically possible
            // I NEVER SEE THIS
            logger.warn("Delete file " + file + " failed", exc);
            try {
                Files.delete(file);
            } catch (IOException e) {
                logger.warn(
                    "Delete file " + file + " failed again", exc);
            }
            return FileVisitResult.CONTINUE;
        }

        @Override
        public FileVisitResult postVisitDirectory(Path dir, IOException exc)
                throws IOException {
            if (exc == null) {
                Files.delete(dir);
                return FileVisitResult.CONTINUE;
            }
            // directory iteration failed; propagate exception
            throw exc;
        }
    });
}

Call:

try {
    removeRecursive(Paths.get(unzipDirPath));
} catch (IOException e) {
    String msg = "Failed to delete folder " + unzipDirPath;
    if (e instanceof java.nio.file.DirectoryNotEmptyException) {
        msg += ". Still contains : ";
        final File[] listFiles = Paths.get(unzipDirPath).toFile().listFiles();
        if (listFiles != null) for (File file : listFiles) {
            msg += file.getAbsolutePath() + "\n";
        }
    }
    log.error(msg, e);
}

Prints (once in 20/40 iterations):

22:03:34.190 [http-bio-8080-exec-47] WARN  g.u.d.m.server.servlets.Controller$1 - Deleting batt
22:03:34.192 [http-bio-8080-exec-47] WARN  g.u.d.m.server.servlets.Controller$1 - DELETED batt
22:03:34.192 [http-bio-8080-exec-47] WARN  g.u.d.m.server.servlets.Controller$1 - Deleting wifi
22:03:34.193 [http-bio-8080-exec-47] WARN  g.u.d.m.server.servlets.Controller$1 - DELETED wifi
22:03:34.196 [http-bio-8080-exec-47] ERROR g.u.d.m.s.s.DataCollectionServlet - Failed to delete folder C:\yada\. Still contains : C:\yada\dir\wifi

java.nio.file.DirectoryNotEmptyException: C:\yada\dir
    at sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:265) ~[na:1.7.0_45]
    at sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103) ~[na:1.7.0_45]
    at java.nio.file.Files.delete(Files.java:1077) ~[na:1.7.0_45]
    at gr.uoa.di.monitoring.server.servlets.Controller$1.postVisitDirectory(Controller.java:128) ~[Controller$1.class:na]
    at gr.uoa.di.monitoring.server.servlets.Controller$1.postVisitDirectory(Controller.java:1) ~[Controller$1.class:na]
    at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:224) ~[na:1.7.0_45]
    at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:199) ~[na:1.7.0_45]
    at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:69) ~[na:1.7.0_45]
    at java.nio.file.Files.walkFileTree(Files.java:2600) ~[na:1.7.0_45]
    at java.nio.file.Files.walkFileTree(Files.java:2633) ~[na:1.7.0_45]
    at gr.uoa.di.monitoring.server.servlets.Controller.removeRecursive(Controller.java:96) ~[Controller.class:na]
    at gr.uoa.di.monitoring.server.servlets.DataCollectionServlet.doPost(DataCollectionServlet.java:153) ~[DataCollectionServlet.class:na]
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:641) [servlet-api.jar:na]
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:722) [servlet-api.jar:na]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305) [catalina.jar:7.0.32]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) [catalina.jar:7.0.32]
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222) [catalina.jar:7.0.32]
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123) [catalina.jar:7.0.32]
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472) [catalina.jar:7.0.32]
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168) [catalina.jar:7.0.32]
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99) [catalina.jar:7.0.32]
    at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:929) [catalina.jar:7.0.32]
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) [catalina.jar:7.0.32]
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407) [catalina.jar:7.0.32]
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1002) [tomcat-coyote.jar:7.0.32]
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:585) [tomcat-coyote.jar:7.0.32]
    at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310) [tomcat-coyote.jar:7.0.32]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_45]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_45]
    at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]

Notice that wifi is reported as deleted - what's even more weird is that sometimes I get :

Failed to delete folder C:\yada. Still contains : C:\yada\dir

java.nio.file.DirectoryNotEmptyException: C:\yada\dir

I tend towards the conclusion that occasionally the deletion takes too long - in other words the problem is that java.nio.file.Files.delete(Path path) does not block (so C:\yada\dir still contains files when its time comes, which sometimes are deleted by the time I stat it). So how am I to workaround this ?

Also : is java.nio.file.Files.delete(Path path) required to throw ? The docs state :

On some operating systems it may not be possible to remove a file when it is open and in use by this Java virtual machine or other programs.

Does not seem to require an exception to be thrown in this case. Is java.nio.file.Files.delete(Path path) required to throw ?

Answer 1

I was having the same problem and it turned out the problem was caused by an unclosed directory file stream somewhere else in the code for the same directory I was deleting stuff from. The stream object returned by:

Files.list(Path)

must be closed, so be sure to use the try-with-resources construct in your code if using that method.

So I don't think it's that the deletion takes too long, I tried a wait before re-attempting directory deletion without any luck. It's most likely that your own program has that resource locked. The result is that the delete call on it doesn't complete although it returns successfully (it looks like Windows will eventually delete the file once your own program frees it) but then of course the containing directory cannot be deleted as that is indeed not yet empty.

Answer 2

I know this is a very old thread - but I had the same problem and it took me quite some time to get it fixed. I think this misbehaviour is caused by a timing issue (looks like it happens on Windows only), so I put a pause into the postVisitDirectory method. That worked and this is what I finally came up with:

A method that does the delete without throwing the DirectoryNotEmptyException:

private boolean isDeleted(Path dir) throws IOException {
    boolean deleted = false;
    try {
        Files.delete(dir);
        deleted = true;
    } catch (DirectoryNotEmptyException e) {
    // happens sometimes if Windows is too slow to remove children of a directory
    deleted = false;
    }
    return deleted;
}

and its use in a loop:

public FileVisitResult postVisitDirectory(Path dir, IOException e) throws IOException {
    if (e == null) {
        int maxTries = 5;
        int count = 0;
        boolean deleted = false;
        do {
            if ((deleted = this.isDeleted(dir))) {
                break;
            } else {
                // wait a bit and try again
                count++;
                try {
                    Thread.sleep(1);
                } catch (InterruptedException e1) {
                    Thread.currentThread().interrupt();
                    break;
                }
            }
        } while (count < maxTries);
        // gone?
        if (!deleted) {
            throw new DirectoryNotEmptyException(dir.toString());
        }
        // go ahead
        return FileVisitResult.CONTINUE;
    }
    throw e;
}

Andy