File.exists() sometimes wrong on Windows 10 with Java 8.191

Question

I have a bunch of unit tests which contain code like:

File file = new File("src/main/java/com/pany/Foo.java");
assertTrue("Missing file: " + file.getAbsolutePath(), file.exists());

This test is suddenly failing when running it with Maven Surefire and -DforkCount=0. With -DforkCount=1, it works.

Things I tried so far:

• The file does exist. Windows Explorer, command line (copy & paste), text editors, Cygwin can all find it and show the contents. That's why I think it's not a permission problem.
• It's not modified by the unit tests or anything else. Git shows no modifications for the last two months.
• I've checked the file system, it's clean.
• I've tried other versions of Java 8, namely 8u171 and 8u181. Same problem.
• I've run Maven from within Cygwin and the command prompt. Same result.
• Reboot :-) No effect :-(

More details:

• When I see this problem, I start to see the "The forked VM terminated without properly saying goodbye. VM crash or System.exit called?" in other projects. That's why I tried forkCount=0 which often helps in this case to find out why the forked VM crashed.
• This has started recently, maybe around the October 2018 update of Windows 10. Before that, the builds were rock solid for about three years. My machine was switched to Windows 10 late 2017, I think.
• I'm using Maven 3.6 and can't easily try an older version because of an important bug that was fixed with it. I did see the VM crash above with Maven 3.5.2 as well.
• It's always the same files which fail (so it's stable).

ulimit (from Cygwin) says:

$ ulimit -a
core file size          (blocks, -c) unlimited
data seg size           (kbytes, -d) unlimited
file size               (blocks, -f) unlimited
open files                      (-n) 256
pipe size            (512 bytes, -p) 8
stack size              (kbytes, -s) 2032
cpu time               (seconds, -t) unlimited
max user processes              (-u) 256
virtual memory          (kbytes, -v) unlimited

I'm wondering if the "open files" limit of 256 only applied to Cygwin processes or whether that's something which Cygwin reads from Windows.

Let me know if you need anything else. I'm running out of ideas what I could try.

Update 1

Bernhard asked me to print absolute names. My answer was that I was already using absolute names but I was wrong. The actual code was:

File file = new File("src/main/java/com/pany/Foo.java");
if (!file.exists()) {
    log.debug("Missing file {}", file.getAbsolutePath());
    ... fail ...
}

... do something with file...

I have now changed this to:

File file = new File("src/main/java/com/pany/Foo.java").getAbsoluteFile();
if (!file.exists()) {
    log.debug("Missing file {}", file);
}

and that fixed the problem. I just can't understand why.

When Maven creates a forked VM to run the tests with Surefire, then it can change the current directory. So in this case, it would make sense that the tests work when forked but fail when running in the same VM (since the VM was created in the root folder of the multi-module build). But why is making the path absolute before the call to exists() fixing the issue?

Answer 1

Some background. Each process has a notion of "current directory". When started from the command line, then it's the directory in which the command was executed. When started from the UI, it's usually the folder in which the program (the .exe file) is.

In the command prompt or BASH, you can change this folder with cd for the process which runs the command prompt.

When Maven builds a multi-module project, it has to change this for each module (so that the relative path src/main/java/ always points to the right place). Unfortunately, Java doesn't have a "set current directory" method anywhere. You can only specify one when creating a new process and you can modify the system property user.dir.

That's why new File("a").exists() and new File("a").getAbsoluteFile().exists() work differently.

The latter will use new File(System.getProperty("user.dir"), "a") to determine the path and the former will use the Windows API function _wgetdcwd (docs) which in turn uses a field of the Windows process to get the current directory - in our case, that's always the folder in which Maven was originally started because Java doesn't update the field in the process when someone changes user.dir and Maven can only change this property to "simulate" changing folders.

WinNTFileSystem_md.c calls fileToNTPath(). That's defined in io_util_md.c and calls pathToNTPath(). For relative paths, it will call currentDirLength() which calls currentDir() which calls _wgetdcwd().

See also:

• https://github.com/openjdk-mirror/jdk7u-jdk/blob/jdk7u6-b08/src/windows/native/java/io/WinNTFileSystem_md.c
• https://github.com/openjdk-mirror/jdk7u-jdk/blob/jdk7u6-b08/src/windows/native/java/io/io_util_md.c

and here is the place where the Surefire plugin modifies the Property user.dir: https://github.com/apache/maven-surefire/blob/56d41b4c903b6c134c5e1a2891f9f08be7e5039f/maven-surefire-common/src/main/java/org/apache/maven/plugin/surefire/AbstractSurefireMojo.java#L1060

When not forking, it's copied into the current VM's System properties: https://github.com/apache/maven-surefire/blob/56d41b4c903b6c134c5e1a2891f9f08be7e5039f/maven-surefire-common/src/main/java/org/apache/maven/plugin/surefire/AbstractSurefireMojo.java#L1133