File sending (large) using Socket Programming

Question

I was wondering if I could get some guidance to solving my problem...

You see I'm trying to get my hands on Socket programming; I managed to create a client and server; the server writes to the client no problem; I've even manage to send files using

byte [] mybytearray  = new byte [(int)myFile.length()];
  FileInputStream fis = new FileInputStream(myFile);

That on the server side. On the client side

byte [] mybytearray  = new byte [filesize];
InputStream is = sock.getInputStream();
FileOutputStream fileos = new FileOutputStream("Filename.dat");
BufferedOutputStream bufferos = new BufferedOutputStream(fileos);
bytesRead = is.read(mybytearray,0,mybytearray.length);
current = bytesRead;


do {
   bytesRead =
      is.read(mybytearray, current, (mybytearray.length-current));
   if(bytesRead >= 0) current += bytesRead;
} while(bytesRead > -1);

bufferos.write(mybytearray, 0 , current);
bufferos.flush();
long end = System.currentTimeMillis();
System.out.println(end-start);
bufferos.close();

My question comes to I'm not able to send large files; I keep on getting "Exception in thread "main" java.lang.OutOfMemoryError: Java heap space"

Any thoughts or directions to how I can manage sending large files to the client from server? I mean sizes of say 600 MB or so.... any thought? Highly appreciated ... thank you

Answer 1

Well, your heap is currently having to try to hold 'mybytearray', which is the size of the file you're trying to receive (presumably). You need to move the bos.write() operation to within the loop, and make 'mybytearray' a fixed size. Here's an example of copying one stream to another that doesn't really care about the size of the data being streamed:

    public static void stream(InputStream in, OutputStream out)
        throws IOException {
    byte[] buf = new byte[1024];
    int bytesRead = 0;

    try {

        while (-1 != (bytesRead = in.read(buf, 0, buf.length))) {
            out.write(buf, 0, bytesRead);
        }

    } catch (IOException e) {
        log.error("Error with streaming op: " + e.getMessage());
        throw (e);
    } finally {
                    try{
           in.close();
           out.flush();
           out.close();
                    } catch (Exception e){}//Ignore
    }
}

Answer 2

You are running out of memory because you are allocating mybytearray to have a length of filesize. One possible solution is to invoke the JVM with a larger heap; e.g. -Xmx1G.

However, another point is whether you need to read the entire file into memory - On the server side this definitely is not the case: You can simply read the file in chunks into a fixed size array and then write these chunks to the output stream. That way your server's memory footprint will not increase when it attempts to send large files back to the client.