In one of my projects I have concurrent write access to one single file within one JRE and want to handle that by first writing to a temporary file and afterwards moving that temp file to the target using an atomic move. I don't care about the order of the write access or such, all I need to guarantee is that any given time the single file is usable. I'm already aware of Files.move and such, my problem is that I had a look at at least one implementation for that method and it raised some doubts about if implementations really guarantee atomic moves. Please look at the following code:
Files.move on GrepCode for OpenJDK
1342 FileSystemProvider provider = provider(source);
1343 if (provider(target) == provider) {
1344 // same provider
1345 provider.move(source, target, options);
1346 } else {
1347 // different providers
1348 CopyMoveHelper.moveToForeignTarget(source, target, options);
1349 }
The problem is that the option ATOMIC_MOVE is not considered in all cases, but the location of the source and target path is the only thing that matters in the first place. That's not what I want and how I understand the documentation:
If the move cannot be performed as an atomic file system operation then AtomicMoveNotSupportedException is thrown. This can arise, for example, when the target location is on a different FileStore and would require that the file be copied, or target location is associated with a different provider to this object.
The above code clearly violates that documentation because it falls back to a copy-delete-strategy without recognizing ATOMIC_MOVE at all. An exception would be perfectly OK in my case, because with that a hoster of our service could change his setup to use only one filesystem which supports atomic moves, as that's what we expect in the system requirements anyway. What I don't want to deal with is things silently failing just because an implementation uses a copy-delete-strategy which may result in data corruption in the target file. So, from my understanding it is simply not safe to rely on Files.move for atomic operations, because it doesn't always fail if those are not supported, but implementations may fall back to a copy-delete-strategy.
Is such behaviour a bug in the implementation and needs to get filed or does the documentation allow such behaviour and I'm understanding it wrong? Does it make any difference at all if I now already know that such maybe broken implementations are used out there? I would need to synchronize the write access on my own in that case...
ATOMIC_MOVE – Performs the move as an atomic file operation. If the file system does not support an atomic move, an exception is thrown. With an ATOMIC_MOVE you can move a file into a directory and be guaranteed that any process watching the directory accesses a complete file.
Two ways to achieve this are described here. The first method utilizes Files package for moving while the other method first copies the file to destination and then deletes the original copy from the source. Using Files. Path move() method: Renaming and moving the file permanently to a new location.
An atomic file operation is an operation that cannot be interrupted or "partially" performed. Either the entire operation is performed or the operation fails.
I came across similar problem to be solved:
Files.move(tmp, out, ATOMIC_MOVE, REPLACE_EXISTING);
And it just does not work reliably, at least on windows. Under heavy load reader occasionally gets NoSuchFileException
- this means Files.move
is not that ATOMIC
even on the same file system :(
My env: Windows 10 + java 11.0.12
Here is the code to play with:
import org.junit.Test;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.nio.ByteBuffer;
import java.nio.channels.ByteChannel;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.Set;
import static java.nio.charset.StandardCharsets.UTF_8;
import static java.nio.file.StandardCopyOption.ATOMIC_MOVE;
import static java.nio.file.StandardCopyOption.REPLACE_EXISTING;
import static java.util.Locale.US;
public class SomeTest {
static int nWrite = 0;
static int nRead = 0;
static int cErrors = 0;
static boolean writeFinished;
static boolean useFileChannels = true;
static String filePath = "c:/temp/test.out";
@Test
public void testParallelFileAccess() throws Exception {
new Writer().start();
new Reader().start();
while( !writeFinished ) {
Thread.sleep(10);
}
System.out.println("cErrors: " + cErrors);
}
static class Writer extends Thread {
public Writer() {
setDaemon(true);
}
@Override
public void run() {
File outFile = new File("c:/temp/test.out");
File outFileTmp = new File(filePath + "tmp");
byte[] bytes = "test".getBytes(UTF_8);
for( nWrite = 1; nWrite <= 100000; nWrite++ ) {
if( (nWrite % 1000) == 0 )
System.out.println("nWrite: " + nWrite + ", cReads: " + nRead);
try( FileOutputStream fos = new FileOutputStream(outFileTmp) ) {
fos.write(bytes);
}
catch( Exception e ) {
logException("write", e);
}
int maxAttemps = 10;
for( int i = 0; i <= maxAttemps; i++ ) {
try {
Files.move(outFileTmp.toPath(), outFile.toPath(), ATOMIC_MOVE, REPLACE_EXISTING);
break;
}
catch( IOException e ) {
try {
Thread.sleep(1);
}
catch( InterruptedException ex ) {
break;
}
if( i == maxAttemps )
logException("move", e);
}
}
}
System.out.println("Write finished ...");
writeFinished = true;
}
}
static class Reader extends Thread {
public Reader() {
setDaemon(true);
}
@Override
public void run() {
File inFile = new File(filePath);
Path inPath = inFile.toPath();
byte[] bytes = new byte[100];
ByteBuffer buffer = ByteBuffer.allocateDirect(100);
try { Thread.sleep(100); } catch( InterruptedException e ) { }
for( nRead = 0; !writeFinished; nRead++ ) {
if( useFileChannels ) {
try ( ByteChannel channel = Files.newByteChannel(inPath, Set.of()) ) {
channel.read(buffer);
}
catch( Exception e ) {
logException("read", e);
}
}
else {
try( InputStream fis = Files.newInputStream(inFile.toPath()) ) {
fis.read(bytes);
}
catch( Exception e ) {
logException("read", e);
}
}
}
}
}
private static void logException(String action, Exception e) {
cErrors++;
System.err.printf(US, "%s: %s - wr=%s, rd=%s:, %s%n", cErrors, action, nWrite, nRead, e);
}
}
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With