Odd behaviour when deleting Files with Files.delete()

Question

Please consider the following example Java class (pom.xml below):

package test.filedelete;  import java.io.ByteArrayInputStream; import java.io.IOException; import java.io.InputStream; import java.io.OutputStream; import java.nio.file.Files; import java.nio.file.NoSuchFileException; import java.nio.file.Path;  import org.apache.commons.io.IOUtils;  public class Main {     public static void main(String[] args) throws IOException     {         byte[] bytes = "testtesttesttesttesttesttesttesttesttest".getBytes();         InputStream is = new ByteArrayInputStream(bytes);          Path tempFileToBeDeleted = Files.createTempFile("test", "");         OutputStream os = Files.newOutputStream(tempFileToBeDeleted);         IOUtils.copy(is, os);          deleteAndCheck(tempFileToBeDeleted);          // breakpoint 1         System.out.println("\nClosing stream\n");          os.close();          deleteAndCheck(tempFileToBeDeleted);     }      private static void deleteAndCheck(Path file) throws IOException     {         System.out.println("Deleting file: " + file);         try         {             Files.delete(file);         }         catch (NoSuchFileException e)         {             System.out.println("No such file");         }         System.out.println("File really deleted: " + !Files.exists(file));          System.out.println("Recreating deleted file ...");         try         {             Files.createFile(file);             System.out.println("Recreation successful");         }         catch (IOException e)         {             System.out.println("Recreation not possible, exception: " + e.getClass().getName());         }     } } 

I write to a FileOutputStream and try to delete the file afterwards without closing the Stream first. This was my original problem, and of course wrong, but it leads to some strange observations.

When you run the main Method on Windows 7 it produces the following output:

Deleting file: C:\Users\MSCHAE~1\AppData\Local\Temp\test6100073603559201768 File really deleted: true Recreating deleted file ... Recreation not possible, exception: java.nio.file.AccessDeniedException  Closing stream  Deleting file: C:\Users\MSCHAE~1\AppData\Local\Temp\test6100073603559201768 No such file File really deleted: true Recreating deleted file ... Recreation successful 

• Why does the first call to Files.delete() not throw an exception?
• Why does the following call to Files.exist() return false?
• Why is it not possible to create the file anew?

Regarding the last question I noticed that the file is still visible in the Explorer when you stop at breakpoint 1. When you terminate the JVM then, the file will be deleted anyway. After closing the stream deleteAndCheck() works as expected.

It seems to me that the deletion is not propagated to the OS before closing the stream and the Files API does not reflect this properly.

Can someone explain exactly what's happening here?

pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"     xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">     <modelVersion>4.0.0</modelVersion>     <groupId>test</groupId>     <artifactId>filedelete</artifactId>     <version>0.0.1-SNAPSHOT</version>      <dependencies>         <dependency>             <groupId>commons-io</groupId>             <artifactId>commons-io</artifactId>             <version>2.4</version>         </dependency>     </dependencies> </project> 

Update for clarification

The file disappears in the Windows explorer, if the stream is closed AND Files.delete() is called - the last operation triggers -, or if Files.delete() has been called without closing the stream and the JVM is terminated.

Answer 1

Can you delete an open file?

It is perfectly valid to delete the directory entry of a file when the file is opened. In Unix this is the default semantics and Windows behaves similarily as long as FILE_SHARE_DELETE is set on all file handles open to that file.

[Edit: Thanks to @couling for discussions and corrections]

However, there is a slight difference: Unix deletes the file name immediately, while Windows deletes the file name only when the last handle is closed. It however, prevents you from opening a file with the same name until the last handle to the (deleted) file is closed.

Go figure ...

On both systems, however, deleting the file does not necessarily make the file go away, it still occupies space on the disk as long as there is still an open handle to it. Space occupied by the file is only released when the last open handle is closed.

Excursion: Windows

That it is necessary to specify the flag on Windows makes it seem to most people that Windows cannot delete open files, but that's actually not true. That's just the default behaviour.

CreateFile():

Enables subsequent open operations on a file or device to request delete access.

Otherwise, other processes cannot open the file or device if they request delete access.

If this flag is not specified, but the file or device has been opened for delete access, the function fails. Note Delete access allows both delete and rename operations.

DeleteFile():

The DeleteFile function marks a file for deletion on close. Therefore, the file deletion does not occur until the last handle to the file is closed. Subsequent calls to CreateFile to open the file fail with ERROR_ACCESS_DENIED.

Having an open handle to a file with no name is one of the most typical methods of creating unnamed temporary files: Create a new file, open it, delete the file. You now have a handle to a file that nobody else can open. On Unix, the filename is truly gone and on Windows you cannot open a file with the same name.

The question is now:

Does Files.newOutputStream() set FILE_SHARE_DELETE?

Looking at the source, you can see that shareDelete indeed defaults to true. The only way to reset it is to use the non-standard ExtendedOpenOption NOSHARE_DELETE.

So yes, you can delete opened files in Java unless they are explicitly locked.

Why can't I re-create the deleted file?

The answer to that is hidden in the documentation of DeleteFile() above: The file is only marked for deletion, the file is still there. On Windows, you cannot create a file with the name of a file marked for deletion until the file is properly deleted, i.e. all handles to the file are closed.

The possible confusion of mixing name deletion and actual file deletion is probably why Windows disallows deleting open files by default in the first place.

Why does Files.exists() return false?

Files.exists() in the deep end on Windows opens that file at some point and we already know that we cannot re-open a deleted-but-still-open file on Windows.

In detail: Java code calls FileSystemProvider.checkAccess()) with no arguments, which calls WindowsFileSystemProvider.checkReadAccess() which straight away tries to open the file and hence fails. From what I can tell, this is the path taken when you call Files.exist().

There is also another code path that calls GetFileAttributeEx() to retrieve file attributes. Once again, it is not documented what happens when you try to retrieve the attributes of an deleted but not yet removed file, but indeed, you cannot retrieve the file attributes of a file marked for deletion.

Guessing, I'd say that GetFileAttributeEx() calls GetFileInformationByHandle() at some point, which it will never get to because it cannot get a file handle in the first place.

So indeed, after DeleteFile() the file is gone for most practical purposes. It still has a name, however, shows up in directory listings and you cannot open a file with the same name until the original file had all its handles closed.

This behaviour is more or less consistent, because using GetFileAttributes() to check if a file exists is a actually an file accessibility check, which is interpreted as file exists. FindFirstFile() (used by Windows Explorer to determine the file list) finds file names but tells you nothing about accessibility of the names.

Welcome to a few more weird loops in your head.