Java Download massive file giving Connection Shutdown/Reset on internet url after sometime

Question

I am building a swing application to download multiple files over the internet and save to a windows fileshare. I have used SwingWroker which internally uses the ExecutorService which internally queues them and downloads 10 at a time, but for some reason after downloading say 2 - 3 MB of file it stops and moves to next downloading file, They are downloaded in a batch of 10 as SwingWorker has fixed it in number of Threads for the Executor Service.

I have to write these files in a windows file share and I am using nio.FileChannels to do that. There are files ranging from 50-60 each weighing around 300MB - 500MB. The file links are located on a webpage to where I get to by login in using credentials on a login page(with a post request) over the internet before that I specify CookieHandler.setDefault(new CookieManager()) at the beginning and so it behaves like a browser to me.

Another observation is when I download them locally (not to a windows server share) they do work fine.

This is the code I am using

import java.io.File;
import java.io.FileOutputStream;
import java.net.URL;
import java.net.URLConnection;
import java.nio.channels.Channels;
import java.nio.channels.FileChannel;
import java.nio.channels.ReadableByteChannel;

import javax.swing.SwingWorker;

public class DownloadProcess extends SwingWorker<Boolean, String> {

  private String urlPath, filePath;
  public DownloadProcess(String urlPath, String filePath){
    this.urlPath = urlPath;
    this.filePath = filePath;
  }             
  @Override
  protected Boolean doInBackground() {
    boolean taskState = true;
    URLConnection httpConn = null;
    ReadableByteChannel readableByteChannel = null;
    FileOutputStream fileOutputStream = null;
    FileChannel fileOutputChannel = null;
    try{
      //String filePath = "\\\\fileshare.server\\xyz.txt";
      //String urlPath = "http://example.com/anyBigFile.1GB.docx";
      File localFile = new File(filePath);//File share
      boolean itsThere = localFile!=null && localFile.exists();
      long done = itsThere ? localFile.length() : 0;
      URL url = new URL(urlPath);
      httpConn = url.openConnection();
      httpConn.setRequestProperty("Connection", "keep-alive");
      if(itsThere) {
        httpConn.setRequestProperty("Range","bytes="+done+"-");
      }
      readableByteChannel = Channels.newChannel(httpConn.getInputStream());
      fileOutputStream = itsThere ? new FileOutputStream(filePath) : new FileOutputStream(filePath,true);
      fileOutputChannel = fileOutputStream.getChannel();
      for (long position = done, size = httpConn.getContentLength(); position < size && !isCancelled(); ) {
        position += fileOutputChannel.transferFrom(readableByteChannel, position, 1 << 16);
      }
      //done
    }catch(Exception e){
      taskState = false;
      e.printStackTrace();

    }finally{
            //close streams conns etc
    }
    return taskState;
  }

}

This is the error stack trace that I get after 5 - 10 mins of download

/*
  javax.net.ssl.SSLException: Connection has been shutdown: javax.net.ssl.SSLException: java.net.SocketException: Connection reset
  at sun.security.ssl.SSLSocketImpl.checkEOF(Unknown Source)
  at sun.security.ssl.AppInputStream.read(Unknown Source)
  at java.io.BufferedInputStream.read1(Unknown Source)
  at java.io.BufferedInputStream.read(Unknown Source)
  at sun.net.www.MeteredStream.read(Unknown Source)
  at java.io.FilterInputStream.read(Unknown Source)
  at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(Unknown Source)
  at java.nio.channels.Channels$ReadableByteChannelImpl.read(Unknown Source)
  at com.objects.DownloadByteChannel.read(DownloadByteChannel.java:117)
  at sun.nio.ch.FileChannelImpl.transferFromArbitraryChannel(Unknown Source)
  at sun.nio.ch.FileChannelImpl.transferFrom(Unknown Source)
  at com.core.DownloadTask.doInBackground(DownloadTask.java:154)
  at com.core.DownloadTask.doInBackground(DownloadTask.java:59)
  at com.util.ZSwingWorker$1.call(ZSwingWorker.java:286)
  at java.util.concurrent.FutureTask.run(Unknown Source)
  at com.util.ZSwingWorker.run(ZSwingWorker.java:325)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
  at java.lang.Thread.run(Unknown Source)
Caused by: javax.net.ssl.SSLException: java.net.SocketException: Connection reset
  at sun.security.ssl.Alerts.getSSLException(Unknown Source)
  at sun.security.ssl.SSLSocketImpl.fatal(Unknown Source)
  at sun.security.ssl.SSLSocketImpl.fatal(Unknown Source)
  at sun.security.ssl.SSLSocketImpl.handleException(Unknown Source)
  at sun.security.ssl.SSLSocketImpl.handleException(Unknown Source)
  ... 18 more
Caused by: java.net.SocketException: Connection reset
  at java.net.SocketInputStream.read(Unknown Source)
  at java.net.SocketInputStream.read(Unknown Source)
  at sun.security.ssl.InputRecord.readFully(Unknown Source)
  at sun.security.ssl.InputRecord.read(Unknown Source)
  at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source)
  at sun.security.ssl.SSLSocketImpl.readDataRecord(Unknown Source)
  ... 18 more
*/

Usage:

public static void main(String[] args){
  int counter = 1;
  for(String url: urls){  
    new DownloadProcess(url,"\\\\fileshare.server\\xyz"+(counter++)+".txt").execute();
  }      
}

Answer 1

You are going to have to change your connection timeout serverside. I picked up a few links along the way if they are of any importance:

Modify Session Security settings

Lengthening salesforce session timeout

Hope this helps, good luck and let me know :)