Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why does nio.files.copy is a lot slower than nio.fileChannel copy?

I'm beginner Java programmer.
Today, I practiced how to copy file in java and tried to follow this tutorial http://www.journaldev.com/861/4-ways-to-copy-file-in-java
Having finished this tutorial, I ran the JMH Benchmark to check the performance, with a 57MB txt file.
And there is a performance gap between nioFiles and NIOChannel, larger than I expected.

Benchmark                                   Mode  Cnt   Score   Error  Units
CompressTest.fileCopyUsingNIOChannelClass  thrpt   10  22.465 ± 2.996  ops/s
CompressTest.fileCopyWithNIOFiles          thrpt   10   0.843 ± 0.488  ops/s

Here is a code I used

    @Warmup(iterations = 5, time = 1000, timeUnit = TimeUnit.MILLISECONDS)
    @Measurement(iterations = 10, time = 1000, timeUnit = TimeUnit.MILLISECONDS)
    @Fork(1)
    @State(Scope.Benchmark)
    public class CompressTest
    {
    final static Path source = Paths.get("c:/temp/system.out.lambda.txt");
    final static Path target = Paths.get("c:/temp/copied.lambda.txt");

    public static void main(String[] args) throws RunnerException, IOException {
        Main.main(args);
    }

    @Benchmark
    public static void fileCopyWithNIOFiles() throws IOException{
        Files.copy(source, target, StandardCopyOption.REPLACE_EXISTING);
    }
    @Benchmark
    public static void fileCopyUsingNIOChannelClass() throws IOException
        {
            File fileToCopy = new File("c:/temp/system.out.lambda.txt");
            FileInputStream inputStream = new FileInputStream(fileToCopy);
            FileChannel inChannel = inputStream.getChannel();

            File newFile = new File("c:/temp/testcopied.txt");
            FileOutputStream outputStream = new FileOutputStream(newFile);
            FileChannel outChannel = outputStream.getChannel();

            inChannel.transferTo(0, fileToCopy.length(), outChannel);
            inputStream.close();
            outputStream.close();
        }
}

So I want to ask, did I do something wrong? or could you explain why this happend?


Somebody asked, so I tried over the other file. 348MB avi file and following is the result.

Benchmark                                   Mode  Cnt  Score   Error  Units
CompressTest.fileCopyUsingNIOChannelClass  thrpt   10  3.142 ± 0.738  ops/s
CompressTest.fileCopyWithNIOFiles          thrpt   10  1.991 ± 0.350  ops/s

Still file copy with a nioFile is slower than using NIOChannel.


I refreshed everything and just tested once more and this is the reuslt. with a 57MB txt file and set warmup iterations as 10.

Benchmark                                   Mode  Cnt   Score   Error  Units
CompressTest.fileCopyUsingNIOChannelClass  thrpt   10  23.442 ± 3.224  ops/s
CompressTest.fileCopyWithNIOFiles          thrpt   10  12.328 ± 2.128  ops/s

This result is more acceptable than a first one, but still the speed of nioFile copying is nearly half of the NIOChannel.

P.S. : if there is a better way to copy a file, please tell me. I really want to learn more about the java, thanks.

like image 867
Dinwy Avatar asked Dec 14 '22 09:12

Dinwy


1 Answers

TL;DR - This is the caching issue

While running the benchmark open Performance tab in the Task Manager, and you'll see where the difference comes from.

During Files.copy test you'll probably see high disk write speed. But when transferTo test is running the disk will be almost idle! This means that no data is actually written to the device - the copying is performed in memory. And this is obviously much faster.

Java Files.copy method is implemented using CopyFileEx WinAPI function. There is no clear specification how CopyFileEx works internally, but it is observed to do some actual disk I/O.

In turn transferTo does series of ReadFile/WriteFile calls. WriteFile function does not guarantee that the data is immediately written to the disk. It may put the data in OS disk cache and perform real device I/O sometime later in background.

like image 74
apangin Avatar answered Dec 16 '22 21:12

apangin