I have utilized the techniques here and concatenated two 1.5GB files in 70 seconds.
http://nadeausoftware.com/articles/2008/02/java_tip_how_read_files_quickly
My code involved using FileChannels with memory mapping and ByteBuffers with an 8KB buffer size.
How could I improve this speed?
File file = new File(binDirectory + "/donjon.avi");
File oFile = new File(binDirectory + "/donjon2.avi");
FileInputStream is = new FileInputStream(file);
FileOutputStream fos = new FileOutputStream(oFile);
FileChannel f1 = is.getChannel();
FileChannel f2 = fos.getChannel();
f2.transferFrom(f1, 0, f1.size());
f2.transferFrom(f1, f1.size(), f1.size());
f2.close();
f1.close();
The best solution would be implement own Writer which directly uses write(byte[]) method of FileOutputStream which used underlying native writeBytes method . like @DavidMoles said source format of data is also very important in this scenario. If data is already available in bytes write directly to FileOutputSteam.
Process Large File In Chunks (BufferdInputStream) We will use BufferedInputStream stream with the same size buffer as we used for FileChannels, and analyse the results. Next is an example of Reading and Writing Large Files in Chunks using Java BufferedInputStream. And, the performance we see is similar to the Scanner.
To choose the merge option, click the arrow next to the Merge button and select the desired merge option. Once complete, the files are merged. If there are multiple files you want to merge at once, you can select multiple files by holding down the Ctrl and selecting each file you want to merge.
try this
FileChannel c1 = new FileInputStream("1").getChannel();
FileChannel c2 = new FileOutputStream("2", true).getChannel();
c2.transferFrom(c1, c2.size(), c1.size());
javadoc says that FileChannel.transferFrom
is potentially much more efficient than a simple loop that reads from this channel and writes to the target channel. Many operating systems can transfer bytes directly from the filesystem cache to the target channel without actually copying them.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With