Java NIO read large file from inputstream

Question

I want to read a large InputStream and return it as a file. So I need to split InputStream(or I should read InputStream in multiple threads). How can I do this? I'm trying to do something like this:

    URL url = new URL("path");
    URLConnection connection = url.openConnection();
    int fileSize = connection.getContentLength();

    InputStream is = connection.getInputStream();
    ReadableByteChannel rbc1 = Channels.newChannel(is);
    ReadableByteChannel rbc2 = Channels.newChannel(is);

    FileOutputStream fos = new FileOutputStream("file.ext");

    FileChannel fileChannel1 = fos.getChannel();
    FileChannel fileChannel2 = fos.getChannel();
    fileChannel1.transferFrom(rbc1, 0, fileSize/2);
    fileChannel2.transferFrom(rbc2, fileSize/2, fileSize/2);

    fos.close();

But it does not affect on performance.

Answer 1

You can open multiple (HTTP) Connections to the same resource (URL) but use the Range: Header of HTTP to make each stream begin to read at another point. This can actually speed up the data transfer, especially when high latency is an issue. You should not overdo the parallelism, be aware that it puts additional load on the server.

connection1.setRequestProperty("Range", "bytes=0-" + half);
connection2.setRequestProperty("Range", "bytes=" + half+1 +"-");

This can also be used to resume downloads. It needs to be supported by the server. It can announce this with Accept-Ranges: bytesbut does not have to . Be prepared that the first connection might return the whole requested entity (status 200 vs. 206) instead.

You need to read the input streams from the URLConnections in separate threads as this is blocking IO (not sure if the NIO wrapping helps here).